diff --git a/source/00-preface.rst b/source/00-preface.rst index 4662c5b..a6c4398 100644 --- a/source/00-preface.rst +++ b/source/00-preface.rst @@ -1,16 +1,16 @@ Preface ******* -For more than a decade, the free and open source VuFind tool has been providing user-friendly access to information from libraries and other organizations. While the software has a supportive and friendly community and detailed online documentation, it has lacked a comprehensive introductory text to orient new users to its features and options. This book is designed to fill that gap, and help you become comfortable with this flexible application. +For more than a decade, the free and open source VuFind® tool has been providing user-friendly access to information from libraries and other organizations. While the software has a supportive and friendly community and detailed online documentation, it has lacked a comprehensive introductory text to orient new users to its features and options. This book is designed to fill that gap, and help you become comfortable with this flexible application. The book is divided into six parts: -• Part 1 will help you understand the purpose of VuFind, install the software, and load some MARC records into it. +• Part 1 will help you understand the purpose of VuFind®, install the software, and load some MARC records into it. • Part 2 will show you how to do some basic configuration and administration of the software. • Part 3 will show you how to customize the software’s look and feel. • Part 4 addresses non-MARC records: XML indexing and HTML crawling. -• Part 5 discusses VuFind’s tools for combining different types of searches into a single interface. -• Part 6 wraps up the book with some of the background necessary to do custom programming within the VuFind environment. +• Part 5 discusses VuFind®’s tools for combining different types of searches into a single interface. +• Part 6 wraps up the book with some of the background necessary to do custom programming within the VuFind® environment. Most chapters are designed to be reasonably stand-alone and will refer back to relevant sections when necessary, so you can skip around and access content out of order if some topics are more relevant to your needs than others. diff --git a/source/01-introduction_to_vufind.rst b/source/01-introduction_to_vufind.rst index 4339b69..59d7cfe 100644 --- a/source/01-introduction_to_vufind.rst +++ b/source/01-introduction_to_vufind.rst @@ -1,17 +1,17 @@ -################################# -Chapter 1. Introduction to VuFind -################################# +################################## +Chapter 1. Introduction to VuFind® +################################## Learning Objectives ------------------- After reading this chapter, you will understand: -1. Why the VuFind project was created. -2. Key features of the VuFind software. -3. Common reasons for using VuFind. +1. Why the VuFind® project was created. +2. Key features of the VuFind® software. +3. Common reasons for using VuFind®. -1.1 What is (and isn’t) VuFind? +1.1 What is (and isn’t) VuFind®? -------------------------------- 1.1.1 History @@ -19,63 +19,63 @@ _____________ In 2007, libraries faced a problem: their mission was to help their users find information, but the information landscape and user expectations were changing. The Google search engine had already been around for a decade, and its single search box approach was becoming ubiquitous. At the same time, many common library search applications were still built around assumptions inherited from the days of card catalogs and featured user interfaces designed during the very earliest days of the World Wide Web. There was a strong need to provide a more user-friendly search experience for library resources. -A group of software developers and designers at Villanova University decided to address this problem through the creation of a search application built using commonly-available, free, open source components. A prototype was built, and this sparked interest from other libraries around the world. A community of software developers quickly grew around the software, collaboratively growing and improving it. After a few years of experimentation, the first stable release was unveiled in early 2010, and enhanced versions have been released every year since then. While members have come and gone, VuFind’s community still exists today, continuing to develop, update and improve the software. +A group of software developers and designers at Villanova University decided to address this problem through the creation of a search application built using commonly-available, free, open source components. A prototype was built, and this sparked interest from other libraries around the world. A community of software developers quickly grew around the software, collaboratively growing and improving it. After a few years of experimentation, the first stable release was unveiled in early 2010, and enhanced versions have been released every year since then. While members have come and gone, VuFind®’s community still exists today, continuing to develop, update and improve the software. 1.1.2 Features ______________ -VuFind’s primary purpose is to provide a user-friendly search experience for diverse resources. The most common application of VuFind is as a library catalog, but it is also frequently used to search collections of journal articles, archival holdings, and even websites. VuFind offers options for searching locally-managed records and also for connecting to third-party search services, such as OCLC’s WorldCat union catalog, and many commercial “web-scale discovery” services. In cases where many different types of resources are available for searching, VuFind offers multiple options for combining and organizing search results. The common denominator is that, wherever the records come from, VuFind presents them in a consistent user interface, reducing user confusion and centralizing the search experience. +VuFind®’s primary purpose is to provide a user-friendly search experience for diverse resources. The most common application of VuFind® is as a library catalog, but it is also frequently used to search collections of journal articles, archival holdings, and even websites. VuFind® offers options for searching locally-managed records and also for connecting to third-party search services, such as OCLC’s WorldCat union catalog, and many commercial “web-scale discovery” services. In cases where many different types of resources are available for searching, VuFind® offers multiple options for combining and organizing search results. The common denominator is that, wherever the records come from, VuFind® presents them in a consistent user interface, reducing user confusion and centralizing the search experience. -In addition to providing robust search functionality, VuFind is also designed to provide easy access to library users’ accounts. Users can renew books, place holds, and view their fines. As with search, VuFind is designed to provide a simple and consistent user experience; while many different staff-facing systems may be used to actually store and manage this information, VuFind provides a single interface for public consumption. This can be especially valuable for institutions planning on changing their back-end software systems; once VuFind is established as the user interface, changes can be made “behind the scenes” without users noticing any significant difference. +In addition to providing robust search functionality, VuFind® is also designed to provide easy access to library users’ accounts. Users can renew books, place holds, and view their fines. As with search, VuFind® is designed to provide a simple and consistent user experience; while many different staff-facing systems may be used to actually store and manage this information, VuFind® provides a single interface for public consumption. This can be especially valuable for institutions planning on changing their back-end software systems; once VuFind® is established as the user interface, changes can be made “behind the scenes” without users noticing any significant difference. -The VuFind software is designed to be extremely configurable and customizable. In addition to interoperating with a wide range of other systems, most of its behaviors can be adjusted through the editing of text-based configuration files. While it contains a wide range of features, most can be turned on or off to suit local needs. The high level of configurability is designed to minimize the need to write local code, but in cases where deep customization is needed, the software’s architecture is designed to make it easy to override components and add functionality by building small plug-ins. +The VuFind® software is designed to be extremely configurable and customizable. In addition to interoperating with a wide range of other systems, most of its behaviors can be adjusted through the editing of text-based configuration files. While it contains a wide range of features, most can be turned on or off to suit local needs. The high level of configurability is designed to minimize the need to write local code, but in cases where deep customization is needed, the software’s architecture is designed to make it easy to override components and add functionality by building small plug-ins. 1.1.3 Limitations _________________ -While VuFind is extremely powerful and flexible, it is important to understand its scope: it is designed as a user interface layer on top of other systems. It provides search capability for existing records, and it provides friendly access to existing user accounts. It is not designed to create or maintain records, however, nor is it meant to be your primary user management system. For library applications, the expectation is that you have an existing integrated library system or library services platform for managing resources and accounts. For other applications, it is assumed that the records you are indexing are created, managed and stored elsewhere. +While VuFind® is extremely powerful and flexible, it is important to understand its scope: it is designed as a user interface layer on top of other systems. It provides search capability for existing records, and it provides friendly access to existing user accounts. It is not designed to create or maintain records, however, nor is it meant to be your primary user management system. For library applications, the expectation is that you have an existing integrated library system or library services platform for managing resources and accounts. For other applications, it is assumed that the records you are indexing are created, managed and stored elsewhere. -These limitations are intentional: managing records and searching records have different requirements. Systems that attempt to do everything usually have to make compromises in terms of performance or functionality. By separating management tasks from user-facing tasks, you can have a more efficient overall system. VuFind is designed for efficient searching, and it allows other systems to focus on other priorities, like maintaining data integrity and providing record editing interfaces. +These limitations are intentional: managing records and searching records have different requirements. Systems that attempt to do everything usually have to make compromises in terms of performance or functionality. By separating management tasks from user-facing tasks, you can have a more efficient overall system. VuFind® is designed for efficient searching, and it allows other systems to focus on other priorities, like maintaining data integrity and providing record editing interfaces. 1.1.4 Use Cases _______________ -There are many possible reasons for using VuFind; here are a few common situations: +There are many possible reasons for using VuFind®; here are a few common situations: • You have multiple sets of resources (e.g. a library catalog, an institutional repository and a website), and you would like to make them searchable through a single box. • You subscribe to a commercial service (such as a library services platform or a web-scale discovery service), but you want more local control over the user experience than the vendor-provided software offers. • You want to provide a user interface that will remain stable over time, even if you replace some components of your infrastructure (such as migrating from one integrated library system to another). • You have a locally-developed information resource (such as a bibliography or archival collection), and you want to provide a public user interface for the data. -Each of these goals can be accomplished with VuFind using the appropriate configuration options. This text will give you the tools necessary to solve these problems, and many more. +Each of these goals can be accomplished with VuFind® using the appropriate configuration options. This text will give you the tools necessary to solve these problems, and many more. -1.2 Prerequisites for Using and Understanding VuFind ----------------------------------------------------- +1.2 Prerequisites for Using and Understanding VuFind® +----------------------------------------------------- In order to make the most of this book, it is helpful to have some background knowledge in a few areas. While going into depth introducing these topics is beyond the scope of this book, many online tutorials and textbooks are available on each of these subjects, should you wish to familiarize yourself with them before proceeding. -While VuFind can run on a variety of operating systems (see section 2.1 for more details), running it on Linux is the preferred option, and so a basic understanding of the Linux command line, file system and permissions will be extremely helpful. Because most of VuFind’s configuration is accomplished by editing text files, comfort with a text editing tool of some sort – such as nano or vi – is a must; fortunately, a wide variety of user-friendly tools are available. +While VuFind® can run on a variety of operating systems (see section 2.1 for more details), running it on Linux is the preferred option, and so a basic understanding of the Linux command line, file system and permissions will be extremely helpful. Because most of VuFind®’s configuration is accomplished by editing text files, comfort with a text editing tool of some sort – such as nano or vi – is a must; fortunately, a wide variety of user-friendly tools are available. -If you plan to customize VuFind’s look and feel – and most VuFind users will at least want to change some logos and adjust some colors – you should have a basic understanding of web technologies like HTML and CSS. For more advanced customization, an understanding of Javascript programming and CSS pre-processors like LESS or SASS/SCSS may be helpful as well. +If you plan to customize VuFind®’s look and feel – and most VuFind® users will at least want to change some logos and adjust some colors – you should have a basic understanding of web technologies like HTML and CSS. For more advanced customization, an understanding of Javascript programming and CSS pre-processors like LESS or SASS/SCSS may be helpful as well. -For advanced customization of VuFind’s behavior, or for integrating the software with systems that are not already supported, you will need an understanding of the PHP programming language and object-oriented programming. Some of this book’s later chapters (part 6) will assume these basics, though they will go into some depth explaining the Laminas framework that provides structure to VuFind’s software components. You can accomplish a great deal with VuFind without having to do any programming, so you can freely skip these chapters if you are not comfortable with this level of detail; however, if you want to have full control over the application, this background knowledge will prove extremely valuable. +For advanced customization of VuFind®’s behavior, or for integrating the software with systems that are not already supported, you will need an understanding of the PHP programming language and object-oriented programming. Some of this book’s later chapters (part 6) will assume these basics, though they will go into some depth explaining the Laminas framework that provides structure to VuFind®’s software components. You can accomplish a great deal with VuFind® without having to do any programming, so you can freely skip these chapters if you are not comfortable with this level of detail; however, if you want to have full control over the application, this background knowledge will prove extremely valuable. -1.3 VuFind Community +1.3 VuFind® Community ______________________ -One of the advantages of using open source software is that successful applications are supported by a community of users and developers who can often be a valuable resource. VuFind is no exception; it has an active and supportive community which provides several options for communication. Documentation for the software is provided through a searchable wiki (https://vufind.org/wiki). When the documentation does not answer a question, users can ask questions on multiple mailing lists or via a Slack community (see the “Support” page of https://vufind.org for the most up-to-date links). The community also streams a regular, free online Community Call to coordinate development of the software, provide updates on new features, and answer questions; the schedule for this can also be found on the website, and all are welcome to join in. +One of the advantages of using open source software is that successful applications are supported by a community of users and developers who can often be a valuable resource. VuFind® is no exception; it has an active and supportive community which provides several options for communication. Documentation for the software is provided through a searchable wiki (https://vufind.org/wiki). When the documentation does not answer a question, users can ask questions on multiple mailing lists or via a Slack community (see the “Support” page of https://vufind.org for the most up-to-date links). The community also streams a regular, free online Community Call to coordinate development of the software, provide updates on new features, and answer questions; the schedule for this can also be found on the website, and all are welcome to join in. Summary ------- -VuFind is an open source project designed to give libraries (and other cultural heritage institutions) more control over their web-based search and account management experience. Users with a basic understanding of Linux commands and HTML/CSS have a great deal of power to configure and customize the software; PHP programmers can go even further. This book will help provide a roadmap to VuFind’s features and options; the project’s friendly community can help answer questions when they arise. +VuFind® is an open source project designed to give libraries (and other cultural heritage institutions) more control over their web-based search and account management experience. Users with a basic understanding of Linux commands and HTML/CSS have a great deal of power to configure and customize the software; PHP programmers can go even further. This book will help provide a roadmap to VuFind®’s features and options; the project’s friendly community can help answer questions when they arise. Review Questions ---------------- -1. What are some of VuFind’s core features? -2. What are some of VuFind’s limitations, and why do they exist? -3. Where can you go to get help with VuFind? -4. What technologies should you familiarize yourself with to make the most of VuFind? -5. Why might a library wish to install VuFind? +1. What are some of VuFind®’s core features? +2. What are some of VuFind®’s limitations, and why do they exist? +3. Where can you go to get help with VuFind®? +4. What technologies should you familiarize yourself with to make the most of VuFind®? +5. Why might a library wish to install VuFind®? diff --git a/source/02-installing_vufind.rst b/source/02-installing_vufind.rst index a7ef1ff..39e69e5 100644 --- a/source/02-installing_vufind.rst +++ b/source/02-installing_vufind.rst @@ -1,46 +1,46 @@ -############################ -Chapter 2. Installing VuFind -############################ +############################# +Chapter 2. Installing VuFind® +############################# Learning Objectives ------------------- After reading this chapter, you will understand: -• The software requirements for installing VuFind. -• The basic VuFind installation process. -• Alternative options for installing VuFind such as Git and Vagrant, and reasons why you might wish to use them. +• The software requirements for installing VuFind®. +• The basic VuFind® installation process. +• Alternative options for installing VuFind® such as Git and Vagrant, and reasons why you might wish to use them. 2.1 System Requirements ----------------------- -VuFind relies on several other pieces of software to do its work, and you will need to have access to these in order to use it. Fortunately, all of VuFind’s dependencies are free and supported by a wide variety of operating systems. Linux is the preferred operating system for VuFind installation, but the software can also be installed successfully on Windows and MacOS. +VuFind® relies on several other pieces of software to do its work, and you will need to have access to these in order to use it. Fortunately, all of VuFind®’s dependencies are free and supported by a wide variety of operating systems. Linux is the preferred operating system for VuFind® installation, but the software can also be installed successfully on Windows and MacOS. -Many of the installation options described in this chapter (such as the Debian package and Vagrant methods) will automatically install VuFind’s dependencies for you, but some of the more advanced options (such as Git-based installation) may require you to install them yourself. +Many of the installation options described in this chapter (such as the Debian package and Vagrant methods) will automatically install VuFind®’s dependencies for you, but some of the more advanced options (such as Git-based installation) may require you to install them yourself. -In order to use VuFind, you will need to have access to four key components: +In order to use VuFind®, you will need to have access to four key components: • The PHP language. • A relational database for storing user and session information; MySQL or MariaDb are recommended, as they are easier to use for basic use cases, but the software also supports PostgreSQL. -• A web server for exposing the VuFind interface to the Internet; Apache is strongly recommended. -• A Java Development Kit (JDK) for running VuFind’s Solr index, and the SolrMarc software used for indexing MARC records. If you will not be working with MARC records, you can use the lighter-weight Java Resource Environment (JRE), since the JDK is only a requirement of the SolrMarc indexer. If you will not be maintaining a local index (a rare situation, but possible in some cases), you will not need Java at all. +• A web server for exposing the VuFind® interface to the Internet; Apache is strongly recommended. +• A Java Development Kit (JDK) for running VuFind®’s Solr index, and the SolrMarc software used for indexing MARC records. If you will not be working with MARC records, you can use the lighter-weight Java Resource Environment (JRE), since the JDK is only a requirement of the SolrMarc indexer. If you will not be maintaining a local index (a rare situation, but possible in some cases), you will not need Java at all. The process of installing software packages varies significantly from operating system to operating system and is thus beyond the scope of this text, but online documentation on each individual package should be available to guide you. If you stick with the basic recommendations below, however, you will not have to worry about these details. -It is also important to understand that software changes and evolves over time, so not every version of VuFind is compatible with every version of PHP, MySQL, etc. Details about version compatibility can be found on the requirements page of the VuFind website -(https://vufind.org/wiki/installation:requirements). If you use the most recent version of VuFind and the most recent stable or long-term-support version of a widely used Linux distribution (such as Fedora or Ubuntu), you are very likely to end up with a compatible set of components. However, if you are using something very old or very new, there is a possibility that something may not work due to version incompatibility. It is helpful to keep this in mind when selecting your platform for installation, and when troubleshooting problems. +It is also important to understand that software changes and evolves over time, so not every version of VuFind® is compatible with every version of PHP, MySQL, etc. Details about version compatibility can be found on the requirements page of the VuFind® website +(https://vufind.org/wiki/installation:requirements). If you use the most recent version of VuFind® and the most recent stable or long-term-support version of a widely used Linux distribution (such as Fedora or Ubuntu), you are very likely to end up with a compatible set of components. However, if you are using something very old or very new, there is a possibility that something may not work due to version incompatibility. It is helpful to keep this in mind when selecting your platform for installation, and when troubleshooting problems. 2.2 Basic Installation ---------------------- -The easiest way to install VuFind is to use a Debian package downloaded from the VuFind website (https://vufind.org/vufind/downloads.html). Debian packages are a method of distributing software for the Debian family of Linux distributions, which includes the popular Ubuntu flavor. The advantage to using a Debian package is that it will automatically install software prerequisites (PHP, MySQL, etc.) as well as complete some basic setup tasks, allowing you to begin working with the software after running only a few commands. Even if you are not currently working in a Debian environment but wish to evaluate the software, you can create a virtual machine using a tool like VirtualBox, install a clean copy of Ubuntu onto it, and follow these instructions from there. +The easiest way to install VuFind® is to use a Debian package downloaded from the VuFind® website (https://vufind.org/vufind/downloads.html). Debian packages are a method of distributing software for the Debian family of Linux distributions, which includes the popular Ubuntu flavor. The advantage to using a Debian package is that it will automatically install software prerequisites (PHP, MySQL, etc.) as well as complete some basic setup tasks, allowing you to begin working with the software after running only a few commands. Even if you are not currently working in a Debian environment but wish to evaluate the software, you can create a virtual machine using a tool like VirtualBox, install a clean copy of Ubuntu onto it, and follow these instructions from there. -If you wish to install VuFind in a different environment, consult the wiki page described above under “Additional Resources” and follow the instructions there. +If you wish to install VuFind® in a different environment, consult the wiki page described above under “Additional Resources” and follow the instructions there. 2.2.1 Installing the Package ____________________________ -First you need to download the .deb package; you can find the URL for the latest version of VuFind (along with historical releases) at https://vufind.org/vufind/downloads.html, and from a Linux command prompt, you can download it with the wget command; for example: +First you need to download the .deb package; you can find the URL for the latest version of VuFind® (along with historical releases) at https://vufind.org/vufind/downloads.html, and from a Linux command prompt, you can download it with the wget command; for example: .. code-block:: console @@ -53,26 +53,26 @@ Once the file is downloaded it, you can install it with: sudo dpkg -i vufind_7.0.deb -Of course, you should substitute the appropriate filename if you have downloaded a different version of VuFind than the example 7.0). +Of course, you should substitute the appropriate filename if you have downloaded a different version of VuFind® than the example 7.0). -This process will fail with an error, but this is expected: the problem is that you have not yet installed all of the software that VuFind needs; fortunately, this can be automatically fixed with this command: +This process will fail with an error, but this is expected: the problem is that you have not yet installed all of the software that VuFind® needs; fortunately, this can be automatically fixed with this command: .. code-block:: console sudo apt-get install -f -It will take some time to download and configure all of the components, but when the process completes, VuFind will be installed and ready for configuration! +It will take some time to download and configure all of the components, but when the process completes, VuFind® will be installed and ready for configuration! -Note that in many Debian-flavored distributions, a bit of extra configuration is needed to set up permissions so that VuFind can connect to the MySQL or MariaDB database software. Details vary from version to version; see the “Important Notes” in the Ubuntu installation wiki page for more details: https://vufind.org/wiki/installation:ubuntu +Note that in many Debian-flavored distributions, a bit of extra configuration is needed to set up permissions so that VuFind® can connect to the MySQL or MariaDB database software. Details vary from version to version; see the “Important Notes” in the Ubuntu installation wiki page for more details: https://vufind.org/wiki/installation:ubuntu If you get stuck or encounter a problem that is not addressed by the documentation, remember that you can reach out to the community for support. See https://vufind.org/vufind/support.html for communication options. -Once successful, the Debian package install will have automatically done a few things for you, including building an Apache configuration to make VuFind accessible through a web browser, adjusting file permissions so that VuFind can write to its cache and update its own configuration files, and setting up some useful environment variables ($VUFIND_HOME and $VUFIND_LOCAL_DIR, which will be discussed further in section 3.3 below). There is a bit more manual work for you to do, however. +Once successful, the Debian package install will have automatically done a few things for you, including building an Apache configuration to make VuFind® accessible through a web browser, adjusting file permissions so that VuFind® can write to its cache and update its own configuration files, and setting up some useful environment variables ($VUFIND_HOME and $VUFIND_LOCAL_DIR, which will be discussed further in section 3.3 below). There is a bit more manual work for you to do, however. 2.2.2 Starting Solr ___________________ -VuFind’s default search functionality is powered by Solr, an open source indexing tool (discussed in much more detail in chapter 5). Because of its importance, VuFind’s installation process will complain if your Solr index is not running. If you do not plan to use Solr, you can ignore this message; however, if you want to be sure you see a full screen of success messages, you can start Solr now. This is simply a matter of switching to the VuFind directory and running the appropriate start command: +VuFind®’s default search functionality is powered by Solr, an open source indexing tool (discussed in much more detail in chapter 5). Because of its importance, VuFind®’s installation process will complain if your Solr index is not running. If you do not plan to use Solr, you can ignore this message; however, if you want to be sure you see a full screen of success messages, you can start Solr now. This is simply a matter of switching to the VuFind® directory and running the appropriate start command: .. code-block:: console @@ -86,22 +86,22 @@ If you receive warning messages or have other problems, you may wish to consult 2.2.3 Initial Configuration ___________________________ -Open a web browser, and point it to http://localhost/vufind/Install -- this should open up a web page showing a number of setup steps. (Note that if you are installing VuFind on one computer and accessing a web browser on a different computer, you should replace “localhost” with the hostname of the VuFind system, and make sure that no firewalls are preventing the two machines from communicating over HTTP). +Open a web browser, and point it to http://localhost/vufind/Install -- this should open up a web page showing a number of setup steps. (Note that if you are installing VuFind® on one computer and accessing a web browser on a different computer, you should replace “localhost” with the hostname of the VuFind® system, and make sure that no firewalls are preventing the two machines from communicating over HTTP). For each item showing a “Failed” status, click on it and follow the on-screen instructions to resolve the problem; once an issue is fixed, you can click the “Auto Configure” breadcrumb to return to the list. Some potentially helpful notes: -• As noted earlier, VuFind can connect to a variety of integrated library systems and library services platforms; by default, it simulates this connection with a “Sample” connector that returns fake data. The installer will warn you about this and offer you the option to configure a real ILS driver. If you do not plan to use an ILS at all, you can select the “NoILS” driver (see section 4.5.1.3), which will disable ILS functionality. If you are not ready to make this decision, you can safely ignore it for now; the setting can be easily changed later. -• Setting up VuFind’s database can be the most challenging part of the installation process, because database security settings can prevent the automatic configuration from working. As mentioned above, the wiki installation documentation should have notes on the latest options for working around common problems. -• Once everything is configured correctly, you should change file permissions on your configuration directory so that VuFind can no longer rewrite its own configurations; this will reduce the chances of accidental or malicious damage to your settings. The installer will provide guidance on how to do this once configuration is complete. +• As noted earlier, VuFind® can connect to a variety of integrated library systems and library services platforms; by default, it simulates this connection with a “Sample” connector that returns fake data. The installer will warn you about this and offer you the option to configure a real ILS driver. If you do not plan to use an ILS at all, you can select the “NoILS” driver (see section 4.5.1.3), which will disable ILS functionality. If you are not ready to make this decision, you can safely ignore it for now; the setting can be easily changed later. +• Setting up VuFind®’s database can be the most challenging part of the installation process, because database security settings can prevent the automatic configuration from working. As mentioned above, the wiki installation documentation should have notes on the latest options for working around common problems. +• Once everything is configured correctly, you should change file permissions on your configuration directory so that VuFind® can no longer rewrite its own configurations; this will reduce the chances of accidental or malicious damage to your settings. The installer will provide guidance on how to do this once configuration is complete. -Once configuration is completed, you should have a fully functional VuFind instance operating at http://localhost/vufind on your system. Of course, there are no records in the system yet, so every search will come up empty. Chapter 3 will help resolve this problem, but first, it is worth learning about some alternative options for installing and managing VuFind. +Once configuration is completed, you should have a fully functional VuFind® instance operating at http://localhost/vufind on your system. Of course, there are no records in the system yet, so every search will come up empty. Chapter 3 will help resolve this problem, but first, it is worth learning about some alternative options for installing and managing VuFind®. 2.3 Other Installation Options ------------------------------ -While installing VuFind as a package is a reasonably straightforward way to manage the software, it may not be the best way to manage it in the long term, especially if you are a software developer. You may find it preferable to use Git to track changes and more easily perform updates, and you may wish to use Vagrant to quickly test the software’s performance in different environments without having to configure them yourself. This section describes the possible roles of these tools in VuFind installation and management. +While installing VuFind® as a package is a reasonably straightforward way to manage the software, it may not be the best way to manage it in the long term, especially if you are a software developer. You may find it preferable to use Git to track changes and more easily perform updates, and you may wish to use Vagrant to quickly test the software’s performance in different environments without having to configure them yourself. This section describes the possible roles of these tools in VuFind® installation and management. 2.3.1 Git _________ @@ -109,17 +109,17 @@ _________ 2.3.1.1 Introduction to Git ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Git is distributed version control software, which is used by the VuFind community to manage development of the software. Git is a widely-used tool in open source, and a valuable asset if you are a software developer. Even for non-programmers, a basic understanding of Git can be helpful for deployment and upgrading of software. +Git is distributed version control software, which is used by the VuFind® community to manage development of the software. Git is a widely-used tool in open source, and a valuable asset if you are a software developer. Even for non-programmers, a basic understanding of Git can be helpful for deployment and upgrading of software. The “version control” portion of “distributed version control” refers to Git’s primary function: tracking changes in software over time. As programmers add or change functionality, they “commit” these changes to Git’s history. This makes it possible to look back through the development of the software, identifying which programmers made which changes and reading their explanations of why those changes were made. When bugs are found, this makes it possible to identify which versions are affected. When mistakes are made, it is possible to roll them back. The software also supports multiple “branches” containing the code in different states of development; by “checking out” a branch, a Git user can instantly change the files on their disk to reflect a particular version of the code. Branches allow developers to work on multiple features at the same time, and test them independently; when work on a branch is completed, it can be “merged” back into the “master” branch, where the latest version of the code resides. When the code is deemed stable enough for an official release, the appropriate Git commit can be “tagged” with a version number, and these tags can be “checked out” just like branches, making it possible to quickly switch between different versions of the software for the purposes of testing and upgrading. The “distributed” part of “distributed version control” refers to the fact that every user of Git creates their own “clone” or “fork” of the software repository that they are working with. They end up with a full copy of all of the history and changes, to which they can add their own commits, branches and tags. This is a significant difference from earlier version control systems like Subversion, which relied on a single shared server to hold all of the change history, which made it more difficult for large groups of developers to work independently of one another. Git comes with tools for “pushing” and “pulling” changes between repositories, so users can work independently with their local repositories without having to worry about what others are doing, and then they can share their work “upstream” when it is in an appropriately polished state. -2.3.1.2 Installing VuFind with Git -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +2.3.1.2 Installing VuFind® with Git +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -To install VuFind using Git, you first need to clone the official VuFind Git repository. If you wish to install the software in the default /usr/local/vufind directory, you could do it like this: +To install VuFind® using Git, you first need to clone the official VuFind® Git repository. If you wish to install the software in the default /usr/local/vufind directory, you could do it like this: .. code-block:: console @@ -135,11 +135,11 @@ This will create a local clone of the repository and automatically check out the git checkout v7.0 -Git will give you all of VuFInd’s code, but nothing else; you will be responsible for installing all of the software that VuFind depends upon – both the requirements described in section 2.1, as well as the package’s Composer dependencies. +Git will give you all of VuFInd’s code, but nothing else; you will be responsible for installing all of the software that VuFind® depends upon – both the requirements described in section 2.1, as well as the package’s Composer dependencies. -One simple way to install VuFind’s software requirements is to install the Debian package as described above. After the package and its dependencies have been installed, you can empty out the /usr/local/vufind directory and use Git to recreate the files (or you can leave the Debian installation alone, and use Git to install a separate copy of VuFind elsewhere on your system). +One simple way to install VuFind®’s software requirements is to install the Debian package as described above. After the package and its dependencies have been installed, you can empty out the /usr/local/vufind directory and use Git to recreate the files (or you can leave the Debian installation alone, and use Git to install a separate copy of VuFind® elsewhere on your system). -To install VuFind’s Composer dependencies, first install Composer (see https://getcomposer.org for instructions) and then, making sure you are in the directory where VuFind was cloned, run: +To install VuFind®’s Composer dependencies, first install Composer (see https://getcomposer.org for instructions) and then, making sure you are in the directory where VuFind® was cloned, run: .. code-block:: console @@ -150,25 +150,25 @@ To learn more about Composer, see the accompanying sidebar. 2.3.1.2.1 Sidebar: About Composer """"""""""""""""""""""""""""""""" -In open source development, it does not make sense to write new software if there is already a good component that can be reused. Most software packages of any complexity depend on many other projects to perform common tasks, and VuFind is no exception. +In open source development, it does not make sense to write new software if there is already a good component that can be reused. Most software packages of any complexity depend on many other projects to perform common tasks, and VuFind® is no exception. Managing these software dependencies can become complex, because components change over time, and it is important to receive updates to fix bugs while avoiding “backward compatibility breaking” changes that might cause problems. Most modern programming languages use tools to manage this process, and Composer is the preferred tool for PHP. -VuFind includes a file called composer.json, which lists all of VuFind’s dependencies, and the versions of those dependencies that are compatible with the rest of the code. Running the “composer install” command reads this file, downloads all of the relevant packages, and installs them into a subdirectory called “vendor.” +VuFind® includes a file called composer.json, which lists all of VuFind®’s dependencies, and the versions of those dependencies that are compatible with the rest of the code. Running the “composer install” command reads this file, downloads all of the relevant packages, and installs them into a subdirectory called “vendor.” -Most VuFind users do not need to concern themselves with this process, but if you plan to become more involved in the software development process, understanding this will be helpful. +Most VuFind® users do not need to concern themselves with this process, but if you plan to become more involved in the software development process, understanding this will be helpful. -Also note that if you install VuFind from a Debian package, or if you download a .tar.gz or .zip file from the website, the vendor directory is already populated for you, and you will not need to worry about Composer at all; this is only a necessary step when you are installing from Git. +Also note that if you install VuFind® from a Debian package, or if you download a .tar.gz or .zip file from the website, the vendor directory is already populated for you, and you will not need to worry about Composer at all; this is only a necessary step when you are installing from Git. 2.3.1.3 Reasons for Using Git ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ There are several reasons why you may wish to consider using Git, most of which have been alluded to above: -• By creating a local Git clone, you can create a branch representing your installed version of VuFind, and you can commit your local configurations to that branch. This will allow you to document the history of your changes to your settings, identifying when decisions were made, and more easily undoing changes that cause problems. +• By creating a local Git clone, you can create a branch representing your installed version of VuFind®, and you can commit your local configurations to that branch. This will allow you to document the history of your changes to your settings, identifying when decisions were made, and more easily undoing changes that cause problems. • Git’s “reset” function makes it easy to restore the “last known good state” of the software. This gives you the freedom to experiment, knowing that you can easily get back to where you started if something breaks. -• If you plan on managing VuFind on multiple servers (for example, development, staging and production environments), you can create branches for each environment, and merge changes between them. You can use the “push” and “pull” features of Git to deploy changes between servers. -• You can more easily upgrade VuFind by pulling updates from the upstream repository and merging them into your local branches; once workflows are established, this can actually be easier than trying to upgrade Debian packages or manually deploy from .tar.gz or .zip files. Scripting can be used to help automatically upgrade your configurations and custom themes as well (see http://blog.library.villanova.edu/libtech/2015/07/23/automatically-updating-locally-customized-files-with-git/ for more information). -• If you wish to participate in VuFind’s development, using Git is almost a necessity for sharing code with the rest of the community. +• If you plan on managing VuFind® on multiple servers (for example, development, staging and production environments), you can create branches for each environment, and merge changes between them. You can use the “push” and “pull” features of Git to deploy changes between servers. +• You can more easily upgrade VuFind® by pulling updates from the upstream repository and merging them into your local branches; once workflows are established, this can actually be easier than trying to upgrade Debian packages or manually deploy from .tar.gz or .zip files. Scripting can be used to help automatically upgrade your configurations and custom themes as well (see http://blog.library.villanova.edu/libtech/2015/07/23/automatically-updating-locally-customized-files-with-git/ for more information). +• If you wish to participate in VuFind®’s development, using Git is almost a necessity for sharing code with the rest of the community. If you find Git intimidating, you certainly do not need to understand it to make use of any of the other information in this book. However, it is a valuable tool, and one that you should consider investigating in the future. Many books and online resources are available to help explain Git in much greater detail than this small section can manage. @@ -185,17 +185,17 @@ Vagrant allows you to create a file called “Vagrantfile” which defines a bas Manually setting up a VM can be a time-consuming and labor-intensive process; Vagrant makes this mostly automatic. A single command can create and configure a VM, and another command can destroy it when you are finished using it. -2.3.2.2 Using Vagrant to Run VuFind -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +2.3.2.2 Using Vagrant to Run VuFind® +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Using Vagrant to run VuFind is quite simple. No matter what method you used to install VuFind, you will find a Vagrantfile in the directory where the software was installed. You can switch to that directory and run: +Using Vagrant to run VuFind® is quite simple. No matter what method you used to install VuFind®, you will find a Vagrantfile in the directory where the software was installed. You can switch to that directory and run: .. code-block:: console vagrant up -This command will take quite some time the first time you run it, as Vagrant has to download a base image for the operating system that the VM will use, and then go through the process of installing and configuring VuFind. In general, after you have started Vagrant once, starting it again in the future will take less time. +This command will take quite some time the first time you run it, as Vagrant has to download a base image for the operating system that the VM will use, and then go through the process of installing and configuring VuFind®. In general, after you have started Vagrant once, starting it again in the future will take less time. Once Vagrant has finished starting up, you can “log in” to the virtual machine by running: @@ -204,9 +204,9 @@ Once Vagrant has finished starting up, you can “log in” to the virtual machi vagrant ssh -This will take you to a command prompt inside the VM. The /vagrant directory in this context is actually a link to the host machine’s VuFind home directory (usually /usr/local/vufind). There is also a directory called /vufindlocal which will hold the VM’s configuration files, and which will only be visible inside the virtual machine. +This will take you to a command prompt inside the VM. The /vagrant directory in this context is actually a link to the host machine’s VuFind® home directory (usually /usr/local/vufind). There is also a directory called /vufindlocal which will hold the VM’s configuration files, and which will only be visible inside the virtual machine. -While the Vagrant VM is active, you can access its VuFind web interface through http://localhost:4567 on your host machine. This is accomplished through Vagrant’s port remapping, which exposes the VM’s port 80 (the standard port used for sharing HTTP web content) to the custom port of 4567 (to prevent the VM from conflicting with the host machine’s normal operation). +While the Vagrant VM is active, you can access its VuFind® web interface through http://localhost:4567 on your host machine. This is accomplished through Vagrant’s port remapping, which exposes the VM’s port 80 (the standard port used for sharing HTTP web content) to the custom port of 4567 (to prevent the VM from conflicting with the host machine’s normal operation). You can temporarily pause the VM with this command: @@ -237,27 +237,27 @@ When you are completely finished with the machine and no longer wish to use it, There are several reasons that Vagrant can be a useful tool: -• Sometimes, the version of VuFind you want to run may not be compatible with your local machine. For example, your PHP version may be too old. Vagrant will automatically install a compatible operating system, and allow you to experiment with the software without having to change or upgrade your host system. Of course, if you wish to run VuFind in production, you will eventually need to set up a compatible server – using Vagrant for a live system is strongly discouraged – but having the ability to test things without having to wait for full server deployment can save a lot of time. +• Sometimes, the version of VuFind® you want to run may not be compatible with your local machine. For example, your PHP version may be too old. Vagrant will automatically install a compatible operating system, and allow you to experiment with the software without having to change or upgrade your host system. Of course, if you wish to run VuFind® in production, you will eventually need to set up a compatible server – using Vagrant for a live system is strongly discouraged – but having the ability to test things without having to wait for full server deployment can save a lot of time. • You may wish to try a potentially disruptive change – for example, some new custom indexing rules. Using a Vagrant box gives you an environment where you can test the change without risking damage to your host machine, and then throw away the results when you are finished. -• You may wish to test whether VuFind will be compatible with a particular platform. As long as that platform has a Vagrant image available, you can modify the default Vagrantfile to use a different base image, and then see what happens, without having to reinstall an operating system or set up a new machine. +• You may wish to test whether VuFind® will be compatible with a particular platform. As long as that platform has a Vagrant image available, you can modify the default Vagrantfile to use a different base image, and then see what happens, without having to reinstall an operating system or set up a new machine. -In general, most VuFind users will not need to use Vagrant – but when these kinds of use cases come up, it can be a valuable and time-saving resource. +In general, most VuFind® users will not need to use Vagrant – but when these kinds of use cases come up, it can be a valuable and time-saving resource. Additional Resources -------------------- -A video covering many of the topics in this chapter is available through the VuFind website (https://vufind.org/wiki/videos:installation). The installation page of the VuFind wiki (https://vufind.org/wiki/installation) contains more detailed and fully up-to-date, step-by-step instructions for installing VuFind in a variety of environments. If the methods described above were not appropriate for your needs, this information should prove helpful. +A video covering many of the topics in this chapter is available through the VuFind® website (https://vufind.org/wiki/videos:installation). The installation page of the VuFind® wiki (https://vufind.org/wiki/installation) contains more detailed and fully up-to-date, step-by-step instructions for installing VuFind® in a variety of environments. If the methods described above were not appropriate for your needs, this information should prove helpful. Summary ------- -VuFind can be installed in a variety of ways, depending on your needs. For a quick, production-ready deployment, using the Debian package under Linux is a convenient option. More experienced users may prefer to handle the installation themselves using Git, and developers may find Vagrant a convenient way to evaluate and test the software without making any potentially risky changes to real systems. +VuFind® can be installed in a variety of ways, depending on your needs. For a quick, production-ready deployment, using the Debian package under Linux is a convenient option. More experienced users may prefer to handle the installation themselves using Git, and developers may find Vagrant a convenient way to evaluate and test the software without making any potentially risky changes to real systems. Review Questions ---------------- -1. Where can you find the most detailed and up-to-date VuFind installation instructions? +1. Where can you find the most detailed and up-to-date VuFind® installation instructions? 2. What kind of operating system do you need to take advantage of a Debian package installation? -3. Should you use Vagrant to install VuFind in a production environment? Why or why not? -4. What are some advantages of installing VuFind using Git? -5. Why does the VuFind project use Composer? +3. Should you use Vagrant to install VuFind® in a production environment? Why or why not? +4. What are some advantages of installing VuFind® using Git? +5. Why does the VuFind® project use Composer? diff --git a/source/03-indexing_marc_records.rst b/source/03-indexing_marc_records.rst index 675f17b..0bf08c5 100644 --- a/source/03-indexing_marc_records.rst +++ b/source/03-indexing_marc_records.rst @@ -7,23 +7,23 @@ Learning Objectives After reading this chapter, you will understand: -• How to load MARC records into VuFind’s Solr index. -• How VuFind’s configuration files are organized and loaded. -• How to customize VuFind’s MARC indexing process. +• How to load MARC records into VuFind®’s Solr index. +• How VuFind®’s configuration files are organized and loaded. +• How to customize VuFind®’s MARC indexing process. 3.1 Obtaining MARC Records -------------------------- The MARC record has been a long-enduring data exchange format for library records. Discussing MARC records in detail is beyond the scope of this text, but the Library of Congress’ Understanding MARC Bibliographic (https://www.loc.gov/marc/umb/) provides a good introduction. For the purposes of this text, suffice it to say that the format arranges data into numeric fields which are divided into alphanumeric subfields. -VuFind ships with powerful tools for indexing MARC records, making MARC one of the easiest formats to begin working with. Since most library systems are capable of exporting MARC records, if you have an integrated library system, there should be a straightforward way to extract all of its records for loading into VuFind. The details of this process will vary from system to system, but notes for a variety of commonly-used platforms can be found in the wiki (https://vufind.org/wiki/indexing:marc:export_notes). +VuFind® ships with powerful tools for indexing MARC records, making MARC one of the easiest formats to begin working with. Since most library systems are capable of exporting MARC records, if you have an integrated library system, there should be a straightforward way to extract all of its records for loading into VuFind®. The details of this process will vary from system to system, but notes for a variety of commonly-used platforms can be found in the wiki (https://vufind.org/wiki/indexing:marc:export_notes). -If you do not have easy access to your own MARC records but wish to experiment with VuFind, there are also a variety of sample records included with the software under the tests/data directory. These sample records will be used in the examples throughout this chapter. +If you do not have easy access to your own MARC records but wish to experiment with VuFind®, there are also a variety of sample records included with the software under the tests/data directory. These sample records will be used in the examples throughout this chapter. 3.2 Running SolrMarc -------------------- -SolrMarc is an open source project for loading MARC records into Solr (a topic which will be discussed in much more detail in chapter 5). Every VuFind release includes a copy of SolrMarc and scripts for running it, so there is no need to install anything extra to get started. To load records into VuFind, you simply need to run the import-marc.sh script and specify the file containing MARC records (which may be either binary MARC format or MARCXML, as long as its filename ends in an appropriate .mrc or .xml extension). For example: +SolrMarc is an open source project for loading MARC records into Solr (a topic which will be discussed in much more detail in chapter 5). Every VuFind® release includes a copy of SolrMarc and scripts for running it, so there is no need to install anything extra to get started. To load records into VuFind®, you simply need to run the import-marc.sh script and specify the file containing MARC records (which may be either binary MARC format or MARCXML, as long as its filename ends in an appropriate .mrc or .xml extension). For example: .. code-block:: console @@ -35,52 +35,52 @@ Note that your Solr index must be active when you run SolrMarc; see 2.2.2 above Once the indexing process is complete, you should be able to return to http://localhost/vufind in your web browser and submit the search form to see what you have indexed; submitting a blank search is a useful way to retrieve all records from the Solr index. Now that we have seen some records added to the system, let’s back up and look at how they got there. -SolrMarc maps MARC records into Solr using configuration files, and the defaults that ship with VuFind should meet most needs. However, most users will want to customize at least some fields, and in some situations, customizations will be necessary for successful record loading. Before getting into the details of SolrMarc customization, however, it is important to understand how VuFind stores and loads its configuration files. +SolrMarc maps MARC records into Solr using configuration files, and the defaults that ship with VuFind® should meet most needs. However, most users will want to customize at least some fields, and in some situations, customizations will be necessary for successful record loading. Before getting into the details of SolrMarc customization, however, it is important to understand how VuFind® stores and loads its configuration files. -3.3 Understanding VuFind’s “Local Directory” +3.3 Understanding VuFind®’s “Local Directory” -------------------------------------------- -In section 2.2.1, it was mentioned that installing VuFind with a Debian package sets up two environment variables on your system: $VUFIND_HOME and $VUFIND_LOCAL_DIR. $VUFIND_HOME contains the full path to the directory containing the VuFind software (/usr/local/vufind by default); $VUFIND_LOCAL_DIR contains the full path to VuFind’s “local directory” (by default, /usr/local/vufind/local). +In section 2.2.1, it was mentioned that installing VuFind® with a Debian package sets up two environment variables on your system: $VUFIND_HOME and $VUFIND_LOCAL_DIR. $VUFIND_HOME contains the full path to the directory containing the VuFind® software (/usr/local/vufind by default); $VUFIND_LOCAL_DIR contains the full path to VuFind®’s “local directory” (by default, /usr/local/vufind/local). -Throughout the rest of this text, when referring to the VuFind home or local directories in file paths, the shorthand variables will be used – so, for example, $VUFIND_LOCAL_DIR/config means /usr/local/vufind/local/config, and $VUFIND_HOME/config means /usr/local/vufind/config. As long as you have these environment variables set on your system, you can use them in commands. This means that if you choose to install VuFind in a non-default location, you can simply redefine the variables (usually by editing the file /etc/profile.d/vufind.sh), and the commands listed in this book will still work correctly. +Throughout the rest of this text, when referring to the VuFind® home or local directories in file paths, the shorthand variables will be used – so, for example, $VUFIND_LOCAL_DIR/config means /usr/local/vufind/local/config, and $VUFIND_HOME/config means /usr/local/vufind/config. As long as you have these environment variables set on your system, you can use them in commands. This means that if you choose to install VuFind® in a non-default location, you can simply redefine the variables (usually by editing the file /etc/profile.d/vufind.sh), and the commands listed in this book will still work correctly. -The purpose of the local directory is to separate locally managed configuration files and data from the core VuFind code. When VuFind needs to load a configuration, it will first look for that file in the local directory; only if the file is missing from the local directory will VuFind load the default version from the home directory. For example, some of SolrMarc’s settings (discussed in more detail below) are stored in a file called import/import.properties. When you run the import-marc.sh command described above, if there is a file matching $VUFIND_LOCAL_DIR/import/import.properties, it will be loaded; if not, $VUFIND_HOME/import/import.properties will be used instead. +The purpose of the local directory is to separate locally managed configuration files and data from the core VuFind® code. When VuFind® needs to load a configuration, it will first look for that file in the local directory; only if the file is missing from the local directory will VuFind® load the default version from the home directory. For example, some of SolrMarc’s settings (discussed in more detail below) are stored in a file called import/import.properties. When you run the import-marc.sh command described above, if there is a file matching $VUFIND_LOCAL_DIR/import/import.properties, it will be loaded; if not, $VUFIND_HOME/import/import.properties will be used instead. -When configuring VuFind, you should always copy configuration files into the local directory and edit them there; you should NEVER edit core configuration files. +When configuring VuFind®, you should always copy configuration files into the local directory and edit them there; you should NEVER edit core configuration files. When used correctly, the local directory approach has several advantages: • By looking in the local directory, you will know exactly which files you have customized. This can be especially valuable when troubleshooting problems, because you can easily compare your changes against the core defaults to see what has been changed, and you can even temporarily disable customizations by moving or renaming files. -• VuFind’s automated upgrade process takes advantage of the separation between core and local files to help you automatically update your configuration files when you upgrade to a new version of VuFind. -• It is possible to maintain multiple local directories containing different settings, and change VuFind’s behavior by changing the value of the $VUFIND_LOCAL_DIR variable. This can be useful, for example, when several libraries share a central index but want to have different user interfaces. This topic is discussed in much more detail in the wiki, on the Installing Multiple Instances page (https://vufind.org/wiki/installation:installing_multiple_instances). +• VuFind®’s automated upgrade process takes advantage of the separation between core and local files to help you automatically update your configuration files when you upgrade to a new version of VuFind®. +• It is possible to maintain multiple local directories containing different settings, and change VuFind®’s behavior by changing the value of the $VUFIND_LOCAL_DIR variable. This can be useful, for example, when several libraries share a central index but want to have different user interfaces. This topic is discussed in much more detail in the wiki, on the Installing Multiple Instances page (https://vufind.org/wiki/installation:installing_multiple_instances). -The local directory can be a source of confusion for new VuFind users, since it is easy to accidentally edit the wrong version of a file if you are not careful. If you edit a configuration file and changes do not appear to take effect, make sure you are in $VUFIND_LOCAL_DIR and not in $VUFIND_HOME. Fortunately, VuFind does not require you to remember too many rules, but the rule of never editing core files is an important one that, if followed consistently, will make your VuFind experience run smoothly. +The local directory can be a source of confusion for new VuFind® users, since it is easy to accidentally edit the wrong version of a file if you are not careful. If you edit a configuration file and changes do not appear to take effect, make sure you are in $VUFIND_LOCAL_DIR and not in $VUFIND_HOME. Fortunately, VuFind® does not require you to remember too many rules, but the rule of never editing core files is an important one that, if followed consistently, will make your VuFind® experience run smoothly. 3.4 About SolrMarc’s Configuration Files ---------------------------------------- -VuFind’s installation of SolrMarc relies on three key configuration files, all of which have default versions in the $VUFIND_HOME/import directory which you can override in $VUFIND_LOCAL_DIR/import. The first of them, import.properties, will be automatically customized for you with some important file paths as part of the VuFind installation process, so do not be surprised when you find your local import directory already populated with a file. +VuFind®’s installation of SolrMarc relies on three key configuration files, all of which have default versions in the $VUFIND_HOME/import directory which you can override in $VUFIND_LOCAL_DIR/import. The first of them, import.properties, will be automatically customized for you with some important file paths as part of the VuFind® installation process, so do not be surprised when you find your local import directory already populated with a file. 3.4.1 import.properties _______________________ -The import.properties file tells SolrMarc some of the most basic information it needs to function: where its other configuration files are located, the URL where Solr is running, and some advanced preferences. In most cases, the defaults created by VuFind’s installer will work correctly, and there is no need to edit this file. However, if you run Solr in a non-default way, or if you encounter problems with the processing of your MARC file, some of the settings in this file may need to be changed. The file contains comments explaining its contents. +The import.properties file tells SolrMarc some of the most basic information it needs to function: where its other configuration files are located, the URL where Solr is running, and some advanced preferences. In most cases, the defaults created by VuFind®’s installer will work correctly, and there is no need to edit this file. However, if you run Solr in a non-default way, or if you encounter problems with the processing of your MARC file, some of the settings in this file may need to be changed. The file contains comments explaining its contents. 3.4.2 marc.properties _____________________ -The marc.properties file is the key to SolrMarc’s behavior. It provides rules for extracting data elements from MARC records and storing them in named fields in the Solr index. These fields are used by VuFind for searching, faceting and record display; Solr will be displayed in much more detail in chapter 5, but the file should be understandable without detailed knowledge of Solr. +The marc.properties file is the key to SolrMarc’s behavior. It provides rules for extracting data elements from MARC records and storing them in named fields in the Solr index. These fields are used by VuFind® for searching, faceting and record display; Solr will be displayed in much more detail in chapter 5, but the file should be understandable without detailed knowledge of Solr. SolrMarc supports several different types of mappings: • Static text strings: if you always want to set a field to the same value, regardless of the contents of the MARC record, you can assign some double-quoted text to a field name, and SolrMarc will insert that value into every record that it indexes. This is used in the default configuration to set the “building” value of every record to “Library A” as an example. • Field specifications: SolrMarc contains its own special language for selecting MARC fields and subfields. Generally, this consists of number/letter combinations, like 035a to select subfield a of field 035, or 100abcd, to select the contents of the a through d subfields of field 100 as a single value. You can combine several of these selectors with colons to select a list of values from all of the specified fields; for example, 440ap:800abcdfpqt:830ap will select values from the specified subfields of the 440, 800 and 830 fields. -• Custom functions: In some situations, selecting data for indexing requires more complex logic than simply selecting a set of fields and subfields. In these situations, a function can be written in the Java programming language, and this custom logic can be accessed in SolrMarc using the “custom” keyword. SolrMarc itself comes with several custom functions, and VuFind adds more. If you need to, you can also build your own, though that topic is beyond the scope of this book. If you want to examine the code for VuFind’s custom indexing functions, you can find them in the $VUFIND_HOME/import/index_java directory. +• Custom functions: In some situations, selecting data for indexing requires more complex logic than simply selecting a set of fields and subfields. In these situations, a function can be written in the Java programming language, and this custom logic can be accessed in SolrMarc using the “custom” keyword. SolrMarc itself comes with several custom functions, and VuFind® adds more. If you need to, you can also build your own, though that topic is beyond the scope of this book. If you want to examine the code for VuFind®’s custom indexing functions, you can find them in the $VUFIND_HOME/import/index_java directory. SolrMarc also provides a number of modifiers which can be added after field specifications or custom functions, which can filter or change the selected values. A very common one is “first,” which will filter down a set containing multiple values to just one value. This is useful in situations where multiple values may be present, but only one is needed. -This quick summary of SolrMarc functionality is intended to help you read and understand VuFind’s default configurations, but it only scratches the surface of the available functionality. For a much more detailed description of available options and their meanings, you can read the documentation available through SolrMarc’s wiki (https://github.com/solrmarc/solrmarc/wiki). +This quick summary of SolrMarc functionality is intended to help you read and understand VuFind®’s default configurations, but it only scratches the surface of the available functionality. For a much more detailed description of available options and their meanings, you can read the documentation available through SolrMarc’s wiki (https://github.com/solrmarc/solrmarc/wiki). 3.4.3 marc_local.properties ___________________________ @@ -95,7 +95,7 @@ Most users of SolrMarc will want to make a few simple customizations; this secti 3.5.1 Overriding Default Collection, Institution and Building Values ____________________________________________________________________ -As noted above under 3.4.2, VuFind’s default indexing configuration includes some made-up values like “Library A” in the building field. “Catalog” in the collection field and “MyInstitution” in the institution field are other hard-coded values that most users will want to override with more appropriate values. Doing this is quite simple. First, if you do not already have a marc_local.properties file, create one by copying the default version into your local directory: +As noted above under 3.4.2, VuFind®’s default indexing configuration includes some made-up values like “Library A” in the building field. “Catalog” in the collection field and “MyInstitution” in the institution field are other hard-coded values that most users will want to override with more appropriate values. Doing this is quite simple. First, if you do not already have a marc_local.properties file, create one by copying the default version into your local directory: .. code-block:: console @@ -124,15 +124,15 @@ Once you have adjusted the settings to meet your needs, you must reindex all of 3.5.2 Loading ID Values _______________________ -When indexing records, it is very important to make sure that the “id” field is filled in correctly. Every record in VuFind needs to have its own unique ID, and this should correspond with the bibliographic record identifier in your Integrated Library System, assuming that you are using one. This value will be used by VuFind to retrieve availability from your ILS and to construct unique record URLs, and by Solr to tell records apart. If you index two records with the same ID into Solr, the second record will overwrite the first one. This mechanism is what makes reindexing existing records behave correctly, but it can cause strange problems if the “id” field is not set up correctly. +When indexing records, it is very important to make sure that the “id” field is filled in correctly. Every record in VuFind® needs to have its own unique ID, and this should correspond with the bibliographic record identifier in your Integrated Library System, assuming that you are using one. This value will be used by VuFind® to retrieve availability from your ILS and to construct unique record URLs, and by Solr to tell records apart. If you index two records with the same ID into Solr, the second record will overwrite the first one. This mechanism is what makes reindexing existing records behave correctly, but it can cause strange problems if the “id” field is not set up correctly. -In VuFind’s default configuration, the MARC 001 field is used to populate “id.” This will work correctly for many systems, but there are some that place the bibliographic identifier in a different place. For example, some methods of exporting records from the Koha ILS will put the appropriate identifier in field 999, subfield c. Thus, to index these records correctly into VuFind, you would have to establish and edit a local copy of marc_local.properties (as described under 3.5.1 above), and then add the line: +In VuFind®’s default configuration, the MARC 001 field is used to populate “id.” This will work correctly for many systems, but there are some that place the bibliographic identifier in a different place. For example, some methods of exporting records from the Koha ILS will put the appropriate identifier in field 999, subfield c. Thus, to index these records correctly into VuFind®, you would have to establish and edit a local copy of marc_local.properties (as described under 3.5.1 above), and then add the line: .. code-block:: properties id = 999c, first -The “first” modifier is probably not strictly necessary, as no Koha record should be exported with more than one ID value. However, it adds a little bit of extra safety in case of an anomaly; without configuration to limit the id field to only one value, a record with multiple IDs would cause a failure in the indexing process, since VuFind’s Solr configuration is not set up to understand how to process a record with more than one ID. +The “first” modifier is probably not strictly necessary, as no Koha record should be exported with more than one ID value. However, it adds a little bit of extra safety in case of an anomaly; without configuration to limit the id field to only one value, a record with multiple IDs would cause a failure in the indexing process, since VuFind®’s Solr configuration is not set up to understand how to process a record with more than one ID. Because of the special role of IDs in Solr indexing, it is also important to be careful about how you manage your index after changing the way IDs are determined. When you reindex records with different ID values, the new records will not overwrite the old ones, and you may end up with duplicates in your system. It is generally a good idea to empty out your index before reindexing when you change ID rules; the process for resetting a Solr index will be discussed below in section 6.2. @@ -140,19 +140,19 @@ Because of the special role of IDs in Solr indexing, it is also important to be Additional Resources -------------------- -A video covering many of the topics in this chapter is available through the VuFind website (https://vufind.org/wiki/videos:indexing_marc_records). The “Indexing MARC” page of the VuFind wiki (https://vufind.org/wiki/indexing:marc) contains additional details and advice that may be more in-depth and up-to-date than this chapter. +A video covering many of the topics in this chapter is available through the VuFind® website (https://vufind.org/wiki/videos:indexing_marc_records). The “Indexing MARC” page of the VuFind® wiki (https://vufind.org/wiki/indexing:marc) contains additional details and advice that may be more in-depth and up-to-date than this chapter. Summary ------- -SolrMarc provides a fast and powerful way of loading MARC records into your VuFind system, making them searchable by your users. It uses VuFind’s “local directory” mechanism to manage its configuration files. SolrMarc has a flexible built-in language that you can use to specify exactly how your records are mapped into VuFind’s index, and VuFind provides a reasonable default configuration that should provide a solid foundation to build upon. +SolrMarc provides a fast and powerful way of loading MARC records into your VuFind® system, making them searchable by your users. It uses VuFind®’s “local directory” mechanism to manage its configuration files. SolrMarc has a flexible built-in language that you can use to specify exactly how your records are mapped into VuFind®’s index, and VuFind® provides a reasonable default configuration that should provide a solid foundation to build upon. Review Questions ---------------- -1. What is VuFind’s “local directory,” and why should you use it? +1. What is VuFind®’s “local directory,” and why should you use it? 2. What is the difference between marc.properties and marc_local.properties? 3. What will happen if you index two different MARC records that have the same value in the field used as Solr’s unique ID? -4. How can you change the values that display in VuFind’s “Collection” and “Building” facets in the search result screen? +4. How can you change the values that display in VuFind®’s “Collection” and “Building” facets in the search result screen? 5. What do $VUFIND_HOME and $VUFIND_LOCAL_DIR mean? 6. Your ILS places bibliographic identifiers in MARC field 997, subfield c. How do you tell SolrMarc to use this as Solr’s unique ID? diff --git a/source/04-configuring_vufind.rst b/source/04-configuring_vufind.rst index 74d49c9..10d5fae 100644 --- a/source/04-configuring_vufind.rst +++ b/source/04-configuring_vufind.rst @@ -1,13 +1,13 @@ -############################# -Chapter 4. Configuring VuFind -############################# +############################## +Chapter 4. Configuring VuFind® +############################## Learning Objectives ------------------- After reading this chapter, you will understand: -• How to find key VuFind configurations. +• How to find key VuFind® configurations. • How to configure search types. • How to configure facets. • The reason for the existence of additional configuration files. @@ -16,36 +16,36 @@ After reading this chapter, you will understand: 4.1 Introduction and Review --------------------------- -VuFind is designed to be highly configurable. The software tries to offer many options through configuration, so that you can integrate with other systems and adjust functionality and appearance with a minimum of extra coding. +VuFind® is designed to be highly configurable. The software tries to offer many options through configuration, so that you can integrate with other systems and adjust functionality and appearance with a minimum of extra coding. -VuFind’s default configurations can be found in the $VUFIND_HOME/config/vufind directory and can be overridden through VuFind’s local directory. If you are not yet comfortable with the way VuFind’s local directory works, please review section 3.3 before continuing. Remember that files found under $VUFIND_LOCAL_DIR/config/vufind will be loaded when they are present; otherwise, default versions from $VUFIND_HOME/config/vufind will be loaded instead. You should NEVER edit the files under $VUFIND_HOME; always copy them to $VUFIND_LOCAL_DIR first. +VuFind®’s default configurations can be found in the $VUFIND_HOME/config/vufind directory and can be overridden through VuFind®’s local directory. If you are not yet comfortable with the way VuFind®’s local directory works, please review section 3.3 before continuing. Remember that files found under $VUFIND_LOCAL_DIR/config/vufind will be loaded when they are present; otherwise, default versions from $VUFIND_HOME/config/vufind will be loaded instead. You should NEVER edit the files under $VUFIND_HOME; always copy them to $VUFIND_LOCAL_DIR first. -The majority of VuFind’s configuration files use the .ini format, in which settings are divided into bracketed [Sections] containing “key = value” pairs. All of these files can be edited with a regular text editor, and they are full of comments explaining the meanings of settings and providing examples. Browsing through them to get a sense of available options may help you discover useful features you were not previously aware of. +The majority of VuFind®’s configuration files use the .ini format, in which settings are divided into bracketed [Sections] containing “key = value” pairs. All of these files can be edited with a regular text editor, and they are full of comments explaining the meanings of settings and providing examples. Browsing through them to get a sense of available options may help you discover useful features you were not previously aware of. -Comments in .ini files are similar to comments in the SolrMarc properties files (described in section 3.5.1) except that .ini comments start with a semi-colon (;) instead of a pound sign (#). As a result, watch out for settings that start with a semi-colon; these are examples, and will not actually have any effect on VuFind until they are uncommented through the removal of the semi-colon. +Comments in .ini files are similar to comments in the SolrMarc properties files (described in section 3.5.1) except that .ini comments start with a semi-colon (;) instead of a pound sign (#). As a result, watch out for settings that start with a semi-colon; these are examples, and will not actually have any effect on VuFind® until they are uncommented through the removal of the semi-colon. 4.2 config.ini -------------- -VuFind’s primary configuration file is named config.ini. When you run VuFind’s web-based installer (as described in section 2.2), one of its steps will copy $VUFIND_HOME/config/vufind/config.ini into $VUFIND_LOCAL_DIR/config/vufind/ for you, and will adjust some of the settings automatically to reflect details like your instance’s URL, and randomly generated keys for secure data encryption. +VuFind®’s primary configuration file is named config.ini. When you run VuFind®’s web-based installer (as described in section 2.2), one of its steps will copy $VUFIND_HOME/config/vufind/config.ini into $VUFIND_LOCAL_DIR/config/vufind/ for you, and will adjust some of the settings automatically to reflect details like your instance’s URL, and randomly generated keys for secure data encryption. -The config.ini file is the place where VuFind looks for most of its global settings – configurations that impact the software as a whole, as opposed to narrower aspects impacting one integrated system or specialized feature. This file controls which page VuFind loads when a user first accesses it, which theme it uses to present its user interface, how users are logged in, and which Integrated Library System it connects to (if any). A comprehensive discussion of every setting in the file would be prohibitively long (and the comments in the file itself provide the most up-to-date narrative), but being aware of this file as a starting point for configuration will help you find the most important settings. You will often see references to this file in online documentation and elsewhere in this book. +The config.ini file is the place where VuFind® looks for most of its global settings – configurations that impact the software as a whole, as opposed to narrower aspects impacting one integrated system or specialized feature. This file controls which page VuFind® loads when a user first accesses it, which theme it uses to present its user interface, how users are logged in, and which Integrated Library System it connects to (if any). A comprehensive discussion of every setting in the file would be prohibitively long (and the comments in the file itself provide the most up-to-date narrative), but being aware of this file as a starting point for configuration will help you find the most important settings. You will often see references to this file in online documentation and elsewhere in this book. 4.3 searches.ini ---------------- -The searches.ini file contains settings related to the way VuFind presents search options related to its Solr index. This file lists legal search options (such as author, title, subject) and sort options (such as date, title, relevance). This is also the place to configure recommendation modules, which provide context-sensitive information accompanying search results (see Chapter 14 for more details). If you want to change default behaviors, rearrange options in drop-down menus, or customize search results screens, this file is a good place to start. Users interested in customizing the way searches are constructed will also need to understand the searchspecs.yaml, which is discussed later in section 5.2. +The searches.ini file contains settings related to the way VuFind® presents search options related to its Solr index. This file lists legal search options (such as author, title, subject) and sort options (such as date, title, relevance). This is also the place to configure recommendation modules, which provide context-sensitive information accompanying search results (see Chapter 14 for more details). If you want to change default behaviors, rearrange options in drop-down menus, or customize search results screens, this file is a good place to start. Users interested in customizing the way searches are constructed will also need to understand the searchspecs.yaml, which is discussed later in section 5.2. -Note that searches.ini is only important if you are using the Solr index for your searches. If you are using a different search method, such as a third-party API, there will be a different configuration file (such as Summon.ini, WorldCat.ini, etc.) which will contain similarly-structured settings that will impact searches from that environment. Most VuFind users incorporate Solr in some way, but if your use case does not require Solr, it is possible that you will never use searches.ini. Even so, understanding its contents will still be useful because whatever backend you use probably relies on most of the same basic concepts. +Note that searches.ini is only important if you are using the Solr index for your searches. If you are using a different search method, such as a third-party API, there will be a different configuration file (such as Summon.ini, WorldCat.ini, etc.) which will contain similarly-structured settings that will impact searches from that environment. Most VuFind® users incorporate Solr in some way, but if your use case does not require Solr, it is possible that you will never use searches.ini. Even so, understanding its contents will still be useful because whatever backend you use probably relies on most of the same basic concepts. 4.4 facets.ini -------------- -The facets.ini file controls VuFind’s faceted search behavior – the lists of values that can be applied to filter search results, as well as pre-filtering options displayed on the advanced search screen, and lists of commonly-found values displayed on the default home page. If you want to add, disable, or rearrange values in any of these places, this file will allow you to effect those changes. +The facets.ini file controls VuFind®’s faceted search behavior – the lists of values that can be applied to filter search results, as well as pre-filtering options displayed on the advanced search screen, and lists of commonly-found values displayed on the default home page. If you want to add, disable, or rearrange values in any of these places, this file will allow you to effect those changes. -As with searches.ini, this file only affects searching of VuFind’s standard Solr index; if you are searching a different service, it will have similar facet settings in its own backend-specific configuration file. +As with searches.ini, this file only affects searching of VuFind®’s standard Solr index; if you are searching a different service, it will have similar facet settings in its own backend-specific configuration file. 4.5 Other Configuration Files ----------------------------- @@ -55,38 +55,38 @@ There are many additional configuration files found in $VUFIND_HOME/config/vufin 4.5.1 ILS Driver Configurations _______________________________ -As noted earlier, VuFind can connect to a wide variety of integrated library systems / library services platforms. Each of these requires special code (called an ILS driver) and configuration (a driver-specific .ini file) to enable communication between VuFind and the target system. There is a [Catalog] section in config.ini which determines which ILS driver VuFind will use. In nearly every case, there is an .ini file in the configuration directory matching the name of the driver, which will contain important settings that will need to be configured – often connection credentials of some sort to grant permission for VuFind’s access, and sometimes additional settings to specify which version of the software is being used, or to enable/disable optional behaviors. +As noted earlier, VuFind® can connect to a wide variety of integrated library systems / library services platforms. Each of these requires special code (called an ILS driver) and configuration (a driver-specific .ini file) to enable communication between VuFind® and the target system. There is a [Catalog] section in config.ini which determines which ILS driver VuFind® will use. In nearly every case, there is an .ini file in the configuration directory matching the name of the driver, which will contain important settings that will need to be configured – often connection credentials of some sort to grant permission for VuFind®’s access, and sometimes additional settings to specify which version of the software is being used, or to enable/disable optional behaviors. In addition to the straightforward ILS drivers that correspond with existing commercial and open source projects, there are also some special ILS drivers. 4.5.1.1 The Demo Driver ^^^^^^^^^^^^^^^^^^^^^^^^ -VuFind’s Demo ILS driver simulates all the functionality of a real ILS driver, but instead of connecting to a real system, it makes up fake results and stores them internally. This is primarily used for software development and testing, since it makes it possible to test and change VuFind’s core behavior without access to a third-party system. It may also be useful for evaluation purposes, to see how a feature will work if you add an ILS in the future. There is no reason to use this driver in a production system, however. +VuFind®’s Demo ILS driver simulates all the functionality of a real ILS driver, but instead of connecting to a real system, it makes up fake results and stores them internally. This is primarily used for software development and testing, since it makes it possible to test and change VuFind®’s core behavior without access to a third-party system. It may also be useful for evaluation purposes, to see how a feature will work if you add an ILS in the future. There is no reason to use this driver in a production system, however. The Demo.ini configuration file controls the behavior of the Demo driver by turning features on or off, or controlling whether or not the driver should simulate system failures for testing purposes. 4.5.1.2 The MultiILS Driver ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The MultiILS driver allows a single copy of VuFind to communicate with multiple ILS drivers. By putting the proper configuration into MultiILS.ini and by indexing records with special ID prefixes, you can set things up so that you can index records from multiple libraries, and have VuFind communicate with the appropriate systems when fetching availability information, placing holds, etc. +The MultiILS driver allows a single copy of VuFind® to communicate with multiple ILS drivers. By putting the proper configuration into MultiILS.ini and by indexing records with special ID prefixes, you can set things up so that you can index records from multiple libraries, and have VuFind® communicate with the appropriate systems when fetching availability information, placing holds, etc. The MultiILS setup is quite complicated, and is only rarely needed (in use cases such as union catalogs), so a detailed discussion is beyond the scope of this book. For more detail, see the appropriate wiki page (https://vufind.org/wiki/configuration:ils:multibackend_driver). 4.5.1.3 The NoILS Driver ^^^^^^^^^^^^^^^^^^^^^^^^^ -The NoILS driver disables some or all of VuFind’s ILS-specific behavior, and it can also replace some functionality normally delegated to an ILS with data retrieval from the Solr index. There are two different use cases for this driver: +The NoILS driver disables some or all of VuFind®’s ILS-specific behavior, and it can also replace some functionality normally delegated to an ILS with data retrieval from the Solr index. There are two different use cases for this driver: -1. By using the loadNoILSOnFailure setting in config.ini, VuFind can be configured to load the NoILS driver instead of the regularly configured driver when a problem is encountered. You can then set the NoILS.ini file into “ils-offline” mode to display a message about temporary unavailability of ILS-related functionality. This is useful to give your users a better experience during planned or unplanned outages of your ILS. -2. If you have no ILS at all, you can select NoILS as your driver and set the NoILS.ini file into “ils-none” mode, and this will ensure that VuFind hides functionality related to the ILS at all times. +1. By using the loadNoILSOnFailure setting in config.ini, VuFind® can be configured to load the NoILS driver instead of the regularly configured driver when a problem is encountered. You can then set the NoILS.ini file into “ils-offline” mode to display a message about temporary unavailability of ILS-related functionality. This is useful to give your users a better experience during planned or unplanned outages of your ILS. +2. If you have no ILS at all, you can select NoILS as your driver and set the NoILS.ini file into “ils-none” mode, and this will ensure that VuFind® hides functionality related to the ILS at all times. -Whether disabling ILS functionality until you are ready to pick a system, or as a safety net in case of failures, the NoILS driver is an important tool to be aware of when configuring VuFind. +Whether disabling ILS functionality until you are ready to pick a system, or as a safety net in case of failures, the NoILS driver is an important tool to be aware of when configuring VuFind®. 4.5.2 Search Backend Configurations ___________________________________ -As noted above, the facets.ini and searches.ini configurations are for Solr, but VuFind can integrate with a wide variety of other search systems. In VuFind terminology, the collection of code used for connecting VuFind to an external search system is referred to as a “search backend,” and most of these backends have corresponding .ini files for storing connection credentials, search/facet preferences, and other service-specific details. +As noted above, the facets.ini and searches.ini configurations are for Solr, but VuFind® can integrate with a wide variety of other search systems. In VuFind® terminology, the collection of code used for connecting VuFind® to an external search system is referred to as a “search backend,” and most of these backends have corresponding .ini files for storing connection credentials, search/facet preferences, and other service-specific details. -Note that some search backends – such as those for Summon and WorldCat – require some connection credentials to be added to config.ini in addition to the backend-specific .ini files; these configurations have not been moved in the interest of backward compatibility with earlier releases of VuFind, but they may be relocated in future to make the configuration layout more uniform. +Note that some search backends – such as those for Summon and WorldCat – require some connection credentials to be added to config.ini in addition to the backend-specific .ini files; these configurations have not been moved in the interest of backward compatibility with earlier releases of VuFind®, but they may be relocated in future to make the configuration layout more uniform. Apart from this one potentially confusing inconsistency, the search backend configurations have been intentionally designed to be as similar to one another as possible. Different systems use different conventions for naming data fields and specifying search types, so the configuration files can vary in significant ways, but the names and behaviors of settings have been kept the same as much as possible. If you learn how to configure one search backend, that knowledge should transfer to configuring the next one, should you need to set up multiple side-by-side search options, or should you migrate from one system to a different one in the future. @@ -95,17 +95,17 @@ The subject of search backends is discussed in greater detail in chapter 15. 4.5.3 Feature-Oriented Configurations _____________________________________ -Some features of VuFind require especially complex configuration and/or are only used in very specialized situations, and putting all of those settings into the main config.ini files would make that file harder to read and work with. Thus, they have been split out into separate files. Some important examples include: combined.ini and searchbox.ini, which will be discussed in more detail in chapter 13; export.ini, which controls the ways in which users can download record data from the system; permissions.ini, which provides rule-based access control over some of VuFind’s features; and sitemap.ini, which controls the creation of sitemap files to assist search engine crawling (see also section 12.2). +Some features of VuFind® require especially complex configuration and/or are only used in very specialized situations, and putting all of those settings into the main config.ini files would make that file harder to read and work with. Thus, they have been split out into separate files. Some important examples include: combined.ini and searchbox.ini, which will be discussed in more detail in chapter 13; export.ini, which controls the ways in which users can download record data from the system; permissions.ini, which provides rule-based access control over some of VuFind®’s features; and sitemap.ini, which controls the creation of sitemap files to assist search engine crawling (see also section 12.2). Additional Resources -------------------- -A video covering most of the information from this chapter is available through the VuFind website +A video covering most of the information from this chapter is available through the VuFind® website (https://vufind.org/wiki/videos:configuring_search_and_facet_settings). For a more comprehensive and up-to-date list of configuration files, see the appropriate wiki page (https://vufind.org/wiki/configuration:files). Summary ------- -VuFind is a heavily configuration-driven piece of software, and it includes what can be an intimidating number of configuration files. However, most users will only need to edit the main config.ini file, and a few additional configurations related to specific systems they integrate with, and particular features that they wish to customize. Because modified configurations need to be stored in VuFind’s local configuration directory, users will always be able to easily focus in on which files are important to their installation. +VuFind® is a heavily configuration-driven piece of software, and it includes what can be an intimidating number of configuration files. However, most users will only need to edit the main config.ini file, and a few additional configurations related to specific systems they integrate with, and particular features that they wish to customize. Because modified configurations need to be stored in VuFind®’s local configuration directory, users will always be able to easily focus in on which files are important to their installation. Review Questions ---------------- diff --git a/source/05-understanding_solr.rst b/source/05-understanding_solr.rst index 478149e..f42dcc2 100644 --- a/source/05-understanding_solr.rst +++ b/source/05-understanding_solr.rst @@ -10,26 +10,26 @@ After reading this chapter, you will understand: • The purpose of Solr, and how it differs from other related tools like databases. • Basic Solr terminology, such as “field type” and “analyzer chain.” • How to interact with Solr, including some basic syntax. -• How to configure VuFind’s use of Solr. +• How to configure VuFind®’s use of Solr. 5.1 Solr Overview ----------------- -Apache Solr is an open source search platform, designed to support fast and flexible full text searching. Since VuFind is primarily concerned with searching, Solr is an ideal engine to drive most of its functionality. While VuFind provides reasonable default settings and tools that save users from having to work directly with Solr, an understanding of how the tool works can be helpful when troubleshooting problems and customizing search behavior. Describing Solr in detail is beyond the scope of this book, but this chapter will provide high-level overviews of some key features; see the “Additional Resources” at the end for some suggestions if you wish to learn more. +Apache Solr is an open source search platform, designed to support fast and flexible full text searching. Since VuFind® is primarily concerned with searching, Solr is an ideal engine to drive most of its functionality. While VuFind® provides reasonable default settings and tools that save users from having to work directly with Solr, an understanding of how the tool works can be helpful when troubleshooting problems and customizing search behavior. Describing Solr in detail is beyond the scope of this book, but this chapter will provide high-level overviews of some key features; see the “Additional Resources” at the end for some suggestions if you wish to learn more. 5.1.1 Indexes vs. Databases ___________________________ -There are many different types of computer systems for storing and retrieving data, and different systems are better suited to different tasks. In fact, while VuFind uses Solr for searching, it also uses a relational database (usually MySQL/MariaDB or PostgreSQL) for storing persistent data like user accounts. Users who have a strong background in databases can find working with Solr confusing at first, so it is important to explain how an index like Solr differs from a relational database like MySQL. +There are many different types of computer systems for storing and retrieving data, and different systems are better suited to different tasks. In fact, while VuFind® uses Solr for searching, it also uses a relational database (usually MySQL/MariaDB or PostgreSQL) for storing persistent data like user accounts. Users who have a strong background in databases can find working with Solr confusing at first, so it is important to explain how an index like Solr differs from a relational database like MySQL. In a relational database, data is stored in a collection of rigidly-structured tables which aim to break content down into the smallest possible pieces. Tables can define relationships with one another so that the small pieces can be recomposed into more complex wholes through querying. This allows data to be stored with very little duplication, which makes maintenance easier. The enforcement of rules helps to prevent certain types of common data entry errors. The clear expression of data types and relationships makes it possible to query the data in complex ways. A relational database is an excellent tool for managing highly structured data. However, it may not be as useful when dealing with extremely heterogeneous information, and searching across many fields and many tables can be slow due to the work involved in fitting all the pieces back together again. An index like Solr differs from a relational database in many significant ways. While it imposes some structure on data, that structure is much less complex: a collection of documents containing named fields. There are fewer rules, and no safeguards to enforce data integrity, because data integrity is not a major concern of an index. Its job is simply to accept queries and find documents that match those queries as quickly and accurately as possible. To meet this goal, it makes a useful trade-off between resource consumption and performance, using significant amounts of memory and disk space to create data structures that help it to very rapidly identify matches. Solr is very good at searching, but it is most powerful when it is used to add search capabilities to data that is created and managed in some other system; while Solr can be used for end-to-end data management, doing so can be problematic, especially when upgrading to a new version of Solr or making changes to the structure of your documents. -In the real world, distinctions between relational databases and indexes can sometimes be blurry; most major relational databases have added full text searching capabilities that resemble Solr, and over time, Solr has made it more feasible (though still not necessarily desirable) to manage data directly in the index. However, even with these caveats, it remains true that each type of system has strengths and weaknesses, and VuFind has been designed to play to the strengths of its components. This is why VuFind only uses Solr for searching, and other tools are used for managing persistent data. +In the real world, distinctions between relational databases and indexes can sometimes be blurry; most major relational databases have added full text searching capabilities that resemble Solr, and over time, Solr has made it more feasible (though still not necessarily desirable) to manage data directly in the index. However, even with these caveats, it remains true that each type of system has strengths and weaknesses, and VuFind® has been designed to play to the strengths of its components. This is why VuFind® only uses Solr for searching, and other tools are used for managing persistent data. 5.1.2 Solr’s Data Model _______________________ @@ -78,16 +78,16 @@ As described in section 4.3, the searches.ini file gives you control over which 5.2.1 Working with YAML _______________________ -As the filename suggests, searchspecs.yaml is a YAML file. YAML is a format designed for storing complex data in a human-readable format. When search configuration functionality was added to VuFind, the .ini format used for most of VuFind’s other configuration files was too simplistic to easily represent the complexity of the available options. Of other commonly-used file formats, XML seemed too verbose, and JSON did not support the ability to insert human-readable comments. Thus, YAML was chosen as a best-of-all-worlds solution. +As the filename suggests, searchspecs.yaml is a YAML file. YAML is a format designed for storing complex data in a human-readable format. When search configuration functionality was added to VuFind®, the .ini format used for most of VuFind®’s other configuration files was too simplistic to easily represent the complexity of the available options. Of other commonly-used file formats, XML seemed too verbose, and JSON did not support the ability to insert human-readable comments. Thus, YAML was chosen as a best-of-all-worlds solution. YAML does have some drawbacks, most particularly a sensitivity to misplaced whitespace. If you edit the searchspecs.yaml file, be very careful that you follow the existing indentation and alignment conventions, since a line out of place can cause the file to be misinterpreted. Fortunately, many of the changes you may wish to make are simply a matter of changing numbers and following existing examples, so it should be possible to avoid pitfalls with a bit of caution. 5.2.2 Anatomy of searchspecs.yaml _________________________________ -The searchspecs.yaml file (found in $VUFIND_HOME/config/vufind along with VuFind’s other configuration files) begins with a long block of comments summarizing the available settings and options; this comment block should always be treated as the most up-to-date documentation on the behavior of the file. +The searchspecs.yaml file (found in $VUFIND_HOME/config/vufind along with VuFind®’s other configuration files) begins with a long block of comments summarizing the available settings and options; this comment block should always be treated as the most up-to-date documentation on the behavior of the file. -Under the documentation comment, there are a number of sections defining VuFind’s “search handlers” – the options that can be configured in searches.ini, which define different types of searches (title, author, subject, etc.) that users can perform. Here is an example, defining VuFind’s “Author” search: +Under the documentation comment, there are a number of sections defining VuFind®’s “search handlers” – the options that can be configured in searches.ini, which define different types of searches (title, author, subject, etc.) that users can perform. Here is an example, defining VuFind®’s “Author” search: .. code-block:: yaml @@ -103,7 +103,7 @@ Under the documentation comment, there are a number of sections defining VuFind - author2_variant DismaxHandler: edismax -The top-level “Author:” begins the block. Everything beneath that is indented by two spaces to show that it is contained within the section – it is this type of indentation that must be carefully maintained to ensure the file is read correctly by the software. Within the block, there are two main settings: “DismaxFields,” which controls relevance ranking (see 5.2.4 below), and DismaxHandler, which in this instance tells VuFind to use Solr’s “Extended DisMax” search functionality. Quite a few other options are supported, some of which will be described in more detail below, and the rest of which can be found described in the aforementioned documentation comment embedded in the searchspecs.yaml file. +The top-level “Author:” begins the block. Everything beneath that is indented by two spaces to show that it is contained within the section – it is this type of indentation that must be carefully maintained to ensure the file is read correctly by the software. Within the block, there are two main settings: “DismaxFields,” which controls relevance ranking (see 5.2.4 below), and DismaxHandler, which in this instance tells VuFind® to use Solr’s “Extended DisMax” search functionality. Quite a few other options are supported, some of which will be described in more detail below, and the rest of which can be found described in the aforementioned documentation comment embedded in the searchspecs.yaml file. 5.2.3 Lucene, DisMax and Extended DisMax ________________________________________ @@ -111,12 +111,12 @@ ________________________________________ You will see the word “DisMax” a lot when reading the file, so it is worth taking a moment to explain it. “DisMax” is short for “Disjunction Max,” which is a powerful Solr query mode which allows the same term to be searched across a number of different fields, with relevance ranking rules applied to bring the “best” matches to the top of the list (see https://cwiki.apache.org/confluence/display/SOLR/DisMax for more details). -Before “DisMax” was introduced, VuFind relied on a much more complicated (and less accurate/reliable) method of generating search queries using the Boolean-based syntax of Solr’s underlying index engine, Lucene. For many years, VuFind operated in a hybrid mode, using DisMax for most searches but switching over to the old method for queries that used features unsupported by DisMax. This old method is still used in VuFind for some very specialized types of searches (see 5.2.6 below for more details), but since the introduction of Solr’s “Extended DisMax” mode, which allows a “best of both worlds” approach combining both DisMax behavior and traditional Boolean logic, VuFind’s search configurations have been significantly streamlined and simplified. Understanding the differences between all of these different search methods is not critical to making the most of VuFind, but knowing a little of this history may make some of the documentation easier to understand. +Before “DisMax” was introduced, VuFind® relied on a much more complicated (and less accurate/reliable) method of generating search queries using the Boolean-based syntax of Solr’s underlying index engine, Lucene. For many years, VuFind® operated in a hybrid mode, using DisMax for most searches but switching over to the old method for queries that used features unsupported by DisMax. This old method is still used in VuFind® for some very specialized types of searches (see 5.2.6 below for more details), but since the introduction of Solr’s “Extended DisMax” mode, which allows a “best of both worlds” approach combining both DisMax behavior and traditional Boolean logic, VuFind®’s search configurations have been significantly streamlined and simplified. Understanding the differences between all of these different search methods is not critical to making the most of VuFind®, but knowing a little of this history may make some of the documentation easier to understand. 5.2.4 Adjusting Relevance _________________________ -Whenever VuFind performs a DisMax search, it looks for user search terms across a set of fields, then does some mathematical analysis on the matches it finds to calculate which documents have the best matches, for the purpose of putting the result set in a useful order. This relevance calculation is informed to a large extent by relative weights applied to the fields being searched. In searchspecs.yaml, the DismaxFields section serves the dual purposes of defining which fields Solr should search across and assigning relative weights to those fields. To return to the Author example from above: +Whenever VuFind® performs a DisMax search, it looks for user search terms across a set of fields, then does some mathematical analysis on the matches it finds to calculate which documents have the best matches, for the purpose of putting the result set in a useful order. This relevance calculation is informed to a large extent by relative weights applied to the fields being searched. In searchspecs.yaml, the DismaxFields section serves the dual purposes of defining which fields Solr should search across and assigning relative weights to those fields. To return to the Author example from above: .. code-block:: yaml @@ -164,7 +164,7 @@ Finally, an advanced option is to provide a boost function using the bf paramete 5.2.5.2 Minimum Matching ^^^^^^^^^^^^^^^^^^^^^^^^ -VuFind is configured that, by default, when a user enters multiple search terms to perform a DisMax search, only records that match ALL of those terms will be returned. However, DisMax can be more tolerant of partial matching – for example, if you want to return records that match 75% of search terms, even if no records match 100% of terms. This extra level of tolerance can be useful, especially when dealing with very long queries. Solr’s mm (“minimum should match”) parameter controls this behavior, and it supports a variety of different scenarios. You can set minimum matching requirements, or you can allow tolerance for certain levels of unmatched clauses. You can even specify different rules for different numbers of incoming terms. See the Solr documentation for a full explanation of how these settings work. Here is a simple example, ensuring that at least 75% of search terms must match: +VuFind® is configured that, by default, when a user enters multiple search terms to perform a DisMax search, only records that match ALL of those terms will be returned. However, DisMax can be more tolerant of partial matching – for example, if you want to return records that match 75% of search terms, even if no records match 100% of terms. This extra level of tolerance can be useful, especially when dealing with very long queries. Solr’s mm (“minimum should match”) parameter controls this behavior, and it supports a variety of different scenarios. You can set minimum matching requirements, or you can allow tolerance for certain levels of unmatched clauses. You can even specify different rules for different numbers of incoming terms. See the Solr documentation for a full explanation of how these settings work. Here is a simple example, ensuring that at least 75% of search terms must match: .. code-block:: yaml @@ -175,7 +175,7 @@ VuFind is configured that, by default, when a user enters multiple search terms ______________ -“Munging” is a slang term for data manipulation, often implying a somewhat crude or rudimentary approach. When working with Solr, it is usually a good idea to let Solr do most of the data manipulation. However, there are rare occasions where it may be useful to have VuFind manipulate user input before submitting it to Solr. For these situations, the searchspecs.yaml “munging” system exists. +“Munging” is a slang term for data manipulation, often implying a somewhat crude or rudimentary approach. When working with Solr, it is usually a good idea to let Solr do most of the data manipulation. However, there are rare occasions where it may be useful to have VuFind® manipulate user input before submitting it to Solr. For these situations, the searchspecs.yaml “munging” system exists. This is probably best explained with another example: @@ -201,14 +201,14 @@ The QueryFields section specifies which field or fields will be searched, and wh Fortunately, because DisMax provides a much simpler configuration and works for the majority of cases, it is rare that users need to work with or understand these older munge-based search types; however, a basic understanding of how to read them may be helpful, especially if you are troubleshooting a search that uses them. Call Number searching is probably the most common remaining use case for this type of search configuration. -5.3 Troubleshooting Solr with VuFind’s Debug Mode +5.3 Troubleshooting Solr with VuFind®’s Debug Mode ------------------------------------------------- -As mentioned earlier, VuFind does the hard work of interacting with Solr for you, and exposes most of the options you will need through configuration files. When you perform a search in VuFind, it translates your query into a Solr query according to the rules defined in its configuration files, then uses that query to retrieve search results, and finally formats those results into the web page that you end up seeing. +As mentioned earlier, VuFind® does the hard work of interacting with Solr for you, and exposes most of the options you will need through configuration files. When you perform a search in VuFind®, it translates your query into a Solr query according to the rules defined in its configuration files, then uses that query to retrieve search results, and finally formats those results into the web page that you end up seeing. -Sometimes, if VuFind does not seem to be finding the results that you think it should, it may be helpful to see exactly what is happening behind the scenes. Fortunately, VuFind includes a debug mode which allows you to see more information about what it is doing. To turn this on, simply edit your local config.ini file, look for the “debug = false” line near the top, and change it to “debug = true.” +Sometimes, if VuFind® does not seem to be finding the results that you think it should, it may be helpful to see exactly what is happening behind the scenes. Fortunately, VuFind® includes a debug mode which allows you to see more information about what it is doing. To turn this on, simply edit your local config.ini file, look for the “debug = false” line near the top, and change it to “debug = true.” -When debug mode is turned on, you will see information boxes scattered around VuFind’s interface full of detailed technical information. When you perform a search, some of these boxes will include full Solr query URLs. Note that, because of the way VuFind’s spell checking feature works, a single search can actually trigger multiple queries against Solr; generally speaking, when looking at VuFind debug output, the first Solr URL is the most important one. +When debug mode is turned on, you will see information boxes scattered around VuFind®’s interface full of detailed technical information. When you perform a search, some of these boxes will include full Solr query URLs. Note that, because of the way VuFind®’s spell checking feature works, a single search can actually trigger multiple queries against Solr; generally speaking, when looking at VuFind® debug output, the first Solr URL is the most important one. For example, try turning on debug mode and then searching for “SOLR” as described in the example in section 5.1. You should see a debug message similar to this: @@ -231,7 +231,7 @@ This is a fairly intimidating block of text, but if you break it apart into chun All of these parameters (and many others) are documented in the online Solr documentation (https://lucene.apache.org/solr/) and clarification can usually be found with the help of the search engine of your choice. -You should be able to copy the URL from this message directly into your web browser to see the Solr response, as well as a more readable summary of the input parameters. (Note that, if the URL starts with http://localhost, you may have to replace “localhost” with the name of the Solr/VuFind server if you are trying to review the results from a different machine – and, of course, access from another machine will only work if firewalls are set up to allow it. For security reasons, cross-machine access to Solr should generally be restricted except when needed for development or troubleshooting). +You should be able to copy the URL from this message directly into your web browser to see the Solr response, as well as a more readable summary of the input parameters. (Note that, if the URL starts with http://localhost, you may have to replace “localhost” with the name of the Solr/VuFind® server if you are trying to review the results from a different machine – and, of course, access from another machine will only work if firewalls are set up to allow it. For security reasons, cross-machine access to Solr should generally be restricted except when needed for development or troubleshooting). In any case, once you have access to Solr query results, you can see exactly what details are coming out of Solr, and you can edit the parameters in the URL to try different searches, sorts, etc. You can add a :code:`&debugQuery=true` parameter on the end of the URL to activate a debug mode that adds an extra section to the Solr response that provides a breakdown of how Solr is processing text and how long each step of the analyzer chain is taking, which can be useful for identifying performance problems. @@ -244,7 +244,7 @@ In any case, once you have access to Solr query results, you can see exactly wha You will also find that if you access just the base part of the URL (which will usually be something like http://localhost:8983/solr), you will find a useful Solr administration panel which lets you explore some of the features of the platform; this also includes a helpful form for building your own queries. If you decide to learn more about Solr using the “Additional Resources” below, you will likely spend a lot of time in this interface. -Note that turning on debug mode can prevent some features of VuFind from working correctly, so it is not a good idea to leave it turned on all the time; it is just intended as a quick-and-easy way to access some technical details like Solr queries. For some more robust alternatives, including logging to files, see the troubleshooting page in VuFind’s wiki (https://vufind.org/wiki/development:troubleshooting). +Note that turning on debug mode can prevent some features of VuFind® from working correctly, so it is not a good idea to leave it turned on all the time; it is just intended as a quick-and-easy way to access some technical details like Solr queries. For some more robust alternatives, including logging to files, see the troubleshooting page in VuFind®’s wiki (https://vufind.org/wiki/development:troubleshooting). Additional Resources @@ -256,11 +256,11 @@ There are several book-length introductions to Solr currently in print as of thi Summary ------- -Solr is an index engine optimized for searching, which is why it serves as the foundation for VuFind’s default search functionality. It provides access to documents by providing field-based searching. A highly configurable search schema and a wealth of search parameters give the administrator a great deal of control over how searches are performed; VuFind exposes much of this functionality through its own configuration files, and its debug mode can help you troubleshoot problems. With a basic understandingof how Solr works and where VuFind’s configuration options can be changed, it should be possible to tune VuFind to meet the needs of local user communities. +Solr is an index engine optimized for searching, which is why it serves as the foundation for VuFind®’s default search functionality. It provides access to documents by providing field-based searching. A highly configurable search schema and a wealth of search parameters give the administrator a great deal of control over how searches are performed; VuFind® exposes much of this functionality through its own configuration files, and its debug mode can help you troubleshoot problems. With a basic understandingof how Solr works and where VuFind®’s configuration options can be changed, it should be possible to tune VuFind® to meet the needs of local user communities. Review Questions ---------------- -1. Why does VuFind use both a relational database and an index? What is the role of each? +1. Why does VuFind® use both a relational database and an index? What is the role of each? 2. What is the benefit of applying the same analyzer chain to indexed text and user search queries? -3. Name three parameters that VuFind passes to Solr when performing searches, and what their purposes are. +3. Name three parameters that VuFind® passes to Solr when performing searches, and what their purposes are. diff --git a/source/06-administering_a_vufind_server.rst b/source/06-administering_a_vufind_server.rst index b041225..ab242fe 100644 --- a/source/06-administering_a_vufind_server.rst +++ b/source/06-administering_a_vufind_server.rst @@ -1,6 +1,6 @@ -######################################## -Chapter 6. Administering a VuFind Server -######################################## +######################################### +Chapter 6. Administering a VuFind® Server +######################################### Learning Objectives --------------------------------------- @@ -8,20 +8,20 @@ Learning Objectives After reading this chapter, you will understand: • How to ensure that necessary processes start up automatically after your server reboots -• How to manage VuFind’s Solr index updates. -• Which maintenance tasks to set up for smooth operation of a VuFind instance +• How to manage VuFind®’s Solr index updates. +• Which maintenance tasks to set up for smooth operation of a VuFind® instance 6.1 Introduction ________________ -It is fairly easy to get a VuFind instance up and running, but there are some potential problems that only become apparent with time. How do you ensure that Solr starts automatically each time you reboot your server? How do you manage rebuilds of your index when you want to change something? How do you keep your index up to date automatically? How do you ensure that VuFind cleans up after itself properly and doesn’t fill up your disk? This chapter answers these and other questions relating to running a production VuFind server for the long term. +It is fairly easy to get a VuFind® instance up and running, but there are some potential problems that only become apparent with time. How do you ensure that Solr starts automatically each time you reboot your server? How do you manage rebuilds of your index when you want to change something? How do you keep your index up to date automatically? How do you ensure that VuFind® cleans up after itself properly and doesn’t fill up your disk? This chapter answers these and other questions relating to running a production VuFind® server for the long term. 6.2 Starting Solr Automatically ------------------------------- -As discussed in 2.2.2, VuFind relies on a running Solr index to allow users to search locally-stored information. While it is simple to manually start the Solr process for testing and development purposes, once you deploy VuFind to production, you will want to ensure that Solr is always running so that users do not experience an interruption to service. An important part of this is ensuring that Solr always starts running when your server boots up, so the service is always available after planned or unplanned server restarts. +As discussed in 2.2.2, VuFind® relies on a running Solr index to allow users to search locally-stored information. While it is simple to manually start the Solr process for testing and development purposes, once you deploy VuFind® to production, you will want to ensure that Solr is always running so that users do not experience an interruption to service. An important part of this is ensuring that Solr always starts running when your server boots up, so the service is always available after planned or unplanned server restarts. -This chapter describes how to set up Solr to start automatically using systemd, the startup system found in the latest releases of most Linux distributions as of this writing. The process is significantly different in Windows and on older Linux distributions; for help with those, see the Starting and Stopping Solr page in the VuFind wiki (https://vufind.org/wiki/administration:starting_and_stopping_solr). +This chapter describes how to set up Solr to start automatically using systemd, the startup system found in the latest releases of most Linux distributions as of this writing. The process is significantly different in Windows and on older Linux distributions; for help with those, see the Starting and Stopping Solr page in the VuFind® wiki (https://vufind.org/wiki/administration:starting_and_stopping_solr). 6.2.1 Creating a Solr User __________________________ @@ -35,7 +35,7 @@ To set this up, you should first create a user with the adduser command. For the sudo adduser solr -Once the user account exists, you should change ownership of VuFind’s Solr directory to the new user so that Solr will have appropriate read/write access to its index files. +Once the user account exists, you should change ownership of VuFind®’s Solr directory to the new user so that Solr will have appropriate read/write access to its index files. .. code-block:: console @@ -45,7 +45,7 @@ Once the user account exists, you should change ownership of VuFind’s Solr dir 6.2.2 Creating the Service File _______________________________ -Programs that can be started or stopped using systemd are known as services. Each is defined by a small configuration file found in /etc/systemd/system which tells the operating system how to manage the program. To manage VuFind’s Solr instance, you can create a file called /etc/systemd/system/vufind.service, containing the following: +Programs that can be started or stopped using systemd are known as services. Each is defined by a small configuration file found in /etc/systemd/system which tells the operating system how to manage the program. To manage VuFind®’s Solr instance, you can create a file called /etc/systemd/system/vufind.service, containing the following: .. code-block:: ini @@ -90,7 +90,7 @@ Finally, to enable the service so that it always starts when your server reboots 6.3 Rebuilding/Resetting the Solr Index --------------------------------------- -There are a variety of reasons that you may eventually want to rebuild your Solr index. When upgrading to a new version of VuFind, it will sometimes be necessary to reindex to reflect changes to VuFind’s Solr schema or updates to the included version of Solr. You may accidentally load bad data into the index and need to create a fresh copy. After months or years of automated synchronization (see 6.4 below), your index may get out of sync with the system that you use to manage your records, and you may wish to rebuild to be sure everything is accurate and up to date. Whatever the reason for rebuilding the index, this section will show you how to do it safely and easily. +There are a variety of reasons that you may eventually want to rebuild your Solr index. When upgrading to a new version of VuFind®, it will sometimes be necessary to reindex to reflect changes to VuFind®’s Solr schema or updates to the included version of Solr. You may accidentally load bad data into the index and need to create a fresh copy. After months or years of automated synchronization (see 6.4 below), your index may get out of sync with the system that you use to manage your records, and you may wish to rebuild to be sure everything is accurate and up to date. Whatever the reason for rebuilding the index, this section will show you how to do it safely and easily. 6.3.1 Resetting the Solr Index ______________________________ @@ -122,7 +122,7 @@ Solr’s index is stored as files on disk, and these files are “portable” When you run a service in production, it is a good practice to maintain a “staging” server that you can use for testing upgrades and customizations before you deploy them to your users. Having a staging server can also be valuable for index regeneration. -Imagine, for example, that you have configured two identical VuFind servers: one for staging, and one for production. As long as both servers are running exactly the same Solr version with exactly the same schema, you could follow these steps to perform a minimal-disruption reindex process: +Imagine, for example, that you have configured two identical VuFind® servers: one for staging, and one for production. As long as both servers are running exactly the same Solr version with exactly the same schema, you could follow these steps to perform a minimal-disruption reindex process: 1. On the staging server, reset your index as described in 6.3.1, and reindex all of your records as described in chapters 3 and 11. @@ -134,31 +134,31 @@ This example procedure still requires a fair amount of manual effort, and is a r 6.4 Automating the Indexing Process ----------------------------------- -If you are using VuFind with an Integrated Library System, it is likely that your records will be changing regularly as new items are cataloged and old ones are weeded. You will want to keep your VuFind index up to date. Unfortunately, every ILS is different, and documenting the automation process for all of them in this book would be impractical. However, this section highlights some of the common tasks and steps you will need to understand to support automation. +If you are using VuFind® with an Integrated Library System, it is likely that your records will be changing regularly as new items are cataloged and old ones are weeded. You will want to keep your VuFind® index up to date. Unfortunately, every ILS is different, and documenting the automation process for all of them in this book would be impractical. However, this section highlights some of the common tasks and steps you will need to understand to support automation. -Many VuFind libraries run a daily cron job which updates the index in the middle of the night, when activity is low. This cron job script should accomplish a few things: +Many VuFind® libraries run a daily cron job which updates the index in the middle of the night, when activity is low. This cron job script should accomplish a few things: 1. Retrieve new records from the ILS. In some cases, it may be possible to use OAI-PMH (see chapter 10); in other situations, it may be necessary to run an ILS-specific command-line script to extract records changed since the last run of the cron job. No matter how the records are obtained, they should be loaded into the index using the standard indexing tool as described in 3.2. -2. Delete removed records from the index. When OAI-PMH is supported, this will be taken care of as part of that process. Otherwise, it may be necessary to obtain a list of deleted records in a different way, and then use VuFind’s $VUFIND_HOME/util/deletes.php script to remove them from the index. +2. Delete removed records from the index. When OAI-PMH is supported, this will be taken care of as part of that process. Otherwise, it may be necessary to obtain a list of deleted records in a different way, and then use VuFind®’s $VUFIND_HOME/util/deletes.php script to remove them from the index. -3. Delete suppressed records from the index. When working with an ILS that allows suppression of bib records, the $VUFIND_HOME/util/suppressed.php script can be used to automatically purge suppressed records from the index, assuming that VuFind’s connector to your ILS supports the necessary functionality. +3. Delete suppressed records from the index. When working with an ILS that allows suppression of bib records, the $VUFIND_HOME/util/suppressed.php script can be used to automatically purge suppressed records from the index, assuming that VuFind®’s connector to your ILS supports the necessary functionality. 4. Optimize the index. After finishing updates to Solr, it is a good idea to run $VUFIND_HOME/util/optimize.php to ensure that your spellcheck index is fully up to date. -5. Regenerate alphabetic browse indexes. If you are using VuFind’s alphabetic browse feature, you should run the $VUFIND_HOME/index-alphabetic-browse.sh script to ensure that browse indexes are up to date. +5. Regenerate alphabetic browse indexes. If you are using VuFind®’s alphabetic browse feature, you should run the $VUFIND_HOME/index-alphabetic-browse.sh script to ensure that browse indexes are up to date. -For more details and some real-world examples, see the Automation page of the VuFind wiki: https://vufind.org/wiki/administration:automation. +For more details and some real-world examples, see the Automation page of the VuFind® wiki: https://vufind.org/wiki/administration:automation. 6.5 Other Important Automated Tasks ----------------------------------- -During the course of day-to-day operation, VuFind generates a significant amount of data that is needed for the short term but which should be cleaned up periodically to save storage space. This information includes user session data, search histories, and authentication tokens. The sections below explain the purpose of this data and how to clean it up when it is no longer needed. +During the course of day-to-day operation, VuFind® generates a significant amount of data that is needed for the short term but which should be cleaned up periodically to save storage space. This information includes user session data, search histories, and authentication tokens. The sections below explain the purpose of this data and how to clean it up when it is no longer needed. 6.5.1 Expiring Searches _______________________ -Every time a user performs a search in VuFind, a row is written to a search table in VuFind’s database. This table allows users to view their search history, and to save some of their searches for long-term use. However, when user sessions expire, many of these search history rows become orphaned and are no longer useful. If left unchecked, these obsolete database rows can grow significantly, wasting large amounts of disk space and impacting system performance. Fortunately, VuFind ships with a simple utility to clear them out. You can simply run: +Every time a user performs a search in VuFind®, a row is written to a search table in VuFind®’s database. This table allows users to view their search history, and to save some of their searches for long-term use. However, when user sessions expire, many of these search history rows become orphaned and are no longer useful. If left unchecked, these obsolete database rows can grow significantly, wasting large amounts of disk space and impacting system performance. Fortunately, VuFind® ships with a simple utility to clear them out. You can simply run: .. code-block:: console @@ -169,7 +169,7 @@ to clear out old searches. It is strongly recommended that you run this command 6.5.2 Expiring Sessions _______________________ -VuFind also uses PHP sessions to store short-term user data (such as their login status). VuFind offers several options for where to store user sessions, such as on disk, in the database, or in a system like Memcached or Redis. The [Session] section of $VUFIND_LOCAL_DIR/config/vufind/config.ini documents all of the options and related settings. +VuFind® also uses PHP sessions to store short-term user data (such as their login status). VuFind® offers several options for where to store user sessions, such as on disk, in the database, or in a system like Memcached or Redis. The [Session] section of $VUFIND_LOCAL_DIR/config/vufind/config.ini documents all of the options and related settings. Like stored searches, session data can build up over time, and while PHP is supposed to help clean this up for you, you may need to supplement PHP’s efforts with some work of your own to be sure things remain under control. If you are using file-based sessions, for example, you may wish to write a cron job to monitor the directory containing the session files and delete those that have not changed in a few days. If you are using database-based sessions, there is a command-line utility similar to the “expire_searches” tool that you can use: @@ -180,7 +180,7 @@ Like stored searches, session data can build up over time, and while PHP is supp 6.5.3 Other Expiry Tools ________________________ -VuFind includes a couple of optional features that may require additional cleanup. +VuFind® includes a couple of optional features that may require additional cleanup. If you use the optional “email authentication” feature (which allows users to log in by clicking on a link in an email sent to them), you may need to periodically clean up the table of pending authentication hashes: @@ -194,7 +194,7 @@ If you use Shibboleth authentication and the “single logout” feature, you ma php $VUFIND_HOME/public/index.php util expire_external_sessions -Over time, it is possible that additional features will be introduced which will require similar cleanup actions. You can always get a summary of VuFind’s available command line utilities by running: +Over time, it is possible that additional features will be introduced which will require similar cleanup actions. You can always get a summary of VuFind®’s available command line utilities by running: .. code-block:: console @@ -205,17 +205,17 @@ Looking through this for additional “expire” actions should reveal whether a 6.6 Managing Code and Configuration ___________________________________ -As you customize and configure VuFind, you will find yourself making changes to dozens of files in multiple directories – configuration files, theme templates, custom code, automation scripts, etc. It is strongly recommended that you consider using a version control system like Git to track all of these things. Git was introduced in section 2.3.1, and if you skipped that section earlier, it may be worth revisiting it now. Even a basic understanding of Git will empower you in several important ways, as detailed in section 2.3.1.3. The value of version control cannot be underestimated; taking the time to learn about it now can save you from much costlier problems down the road. +As you customize and configure VuFind®, you will find yourself making changes to dozens of files in multiple directories – configuration files, theme templates, custom code, automation scripts, etc. It is strongly recommended that you consider using a version control system like Git to track all of these things. Git was introduced in section 2.3.1, and if you skipped that section earlier, it may be worth revisiting it now. Even a basic understanding of Git will empower you in several important ways, as detailed in section 2.3.1.3. The value of version control cannot be underestimated; taking the time to learn about it now can save you from much costlier problems down the road. Additional Resources -------------------- -As noted above, you can find more information about starting Solr automatically on the Starting and Stopping Solr page in the VuFind wiki (https://vufind.org/wiki/administration:starting_and_stopping_solr). You can learn more about automatic index updates on the Automation page of the VuFind wiki (https://vufind.org/wiki/administration:automation). Some of the topics from this chapter are demonstrated in the video available here: https://vufind.org/wiki/videos:administering_a_vufind_server. +As noted above, you can find more information about starting Solr automatically on the Starting and Stopping Solr page in the VuFind® wiki (https://vufind.org/wiki/administration:starting_and_stopping_solr). You can learn more about automatic index updates on the Automation page of the VuFind® wiki (https://vufind.org/wiki/administration:automation). Some of the topics from this chapter are demonstrated in the video available here: https://vufind.org/wiki/videos:administering_a_vufind_server. Summary ------- -Reliably running a VuFind server in production requires some additional configuration and maintenance. By utilizing operating system auto-start features, intelligently managing your Solr indexing processes, and regularly cleaning up expired data, you can ensure that your users have a reliable and uninterrupted search experience. +Reliably running a VuFind® server in production requires some additional configuration and maintenance. By utilizing operating system auto-start features, intelligently managing your Solr indexing processes, and regularly cleaning up expired data, you can ensure that your users have a reliable and uninterrupted search experience. Review Questions diff --git a/source/07-creating_a_vufind_theme.rst b/source/07-creating_a_vufind_theme.rst index cdf5421..063f5a9 100644 --- a/source/07-creating_a_vufind_theme.rst +++ b/source/07-creating_a_vufind_theme.rst @@ -1,33 +1,33 @@ -################################## -Chapter 7. Creating a VuFind Theme -################################## +################################### +Chapter 7. Creating a VuFind® Theme +################################### Learning Objectives ------------------- After reading this chapter, you will understand: -• How to create your own VuFind theme. +• How to create your own VuFind® theme. • How to override view templates. -• How to read and understand the conventions of VuFind’s view templates. -• How to use some of VuFind’s advanced theme configuration options +• How to read and understand the conventions of VuFind®’s view templates. +• How to use some of VuFind®’s advanced theme configuration options 7.1 Introduction to Themes -------------------------- -VuFind is designed using a model-view-controller architecture (as discussed further in section 16.2). One of the key features of this design is that the code for how VuFind does things (the “business logic”) is separated from the code for how VuFind displays things (the “view templates”). This is a powerful separation, since it means that VuFind’s look and feel can be changed independently of its core functionality. It also means that it is possible to provide multiple presentations of the system. +VuFind® is designed using a model-view-controller architecture (as discussed further in section 16.2). One of the key features of this design is that the code for how VuFind® does things (the “business logic”) is separated from the code for how VuFind® displays things (the “view templates”). This is a powerful separation, since it means that VuFind®’s look and feel can be changed independently of its core functionality. It also means that it is possible to provide multiple presentations of the system. -VuFind includes a theme system which makes it easy to bundle all of VuFind’s user-facing pieces (view templates, Javascript code, CSS styles, images) in a single package. This theme system also supports the idea of inheritance, so you can create a “sub-theme” which can override some parts of a parent theme while inheriting other parts as-is. For example, if you like one of VuFind’s default themes but just want to override some CSS styles and the text of the footer, you can easily create a sub-theme that contains a custom CSS file and a custom footer template, and all of the other functionality will carry through from the parent theme. +VuFind® includes a theme system which makes it easy to bundle all of VuFind®’s user-facing pieces (view templates, Javascript code, CSS styles, images) in a single package. This theme system also supports the idea of inheritance, so you can create a “sub-theme” which can override some parts of a parent theme while inheriting other parts as-is. For example, if you like one of VuFind®’s default themes but just want to override some CSS styles and the text of the footer, you can easily create a sub-theme that contains a custom CSS file and a custom footer template, and all of the other functionality will carry through from the parent theme. -VuFind includes a root theme, from which all other themes extend; this theme contains some universally-needed templates (such as pre-formatted email messages, rules for exporting records into bibliographic management systems, help screens, etc.) and some shared fonts and images. The other available themes will depend on the version of VuFind you are using; as web technology evolves quickly, VuFind has kept up with progress by providing several different theme options over the years, leveraging different frameworks. While the names may change over time, the basic principle is that there is always a base theme that contains all of the templates used for rendering VuFind’s many pages, and then there are some sub-themes which apply extra CSS to this foundation in order to provide different stylistic options. You can extend from any of these options to build your own theme; it is simply a matter of which option provides the most work that you can reuse in order to keep your local customizations as small and simple as possible. +VuFind® includes a root theme, from which all other themes extend; this theme contains some universally-needed templates (such as pre-formatted email messages, rules for exporting records into bibliographic management systems, help screens, etc.) and some shared fonts and images. The other available themes will depend on the version of VuFind® you are using; as web technology evolves quickly, VuFind® has kept up with progress by providing several different theme options over the years, leveraging different frameworks. While the names may change over time, the basic principle is that there is always a base theme that contains all of the templates used for rendering VuFind®’s many pages, and then there are some sub-themes which apply extra CSS to this foundation in order to provide different stylistic options. You can extend from any of these options to build your own theme; it is simply a matter of which option provides the most work that you can reuse in order to keep your local customizations as small and simple as possible. -This chapter will show you how to create your own themes, how to understand some of the code you will see in the existing themes, and how to configure VuFind to use different themes. +This chapter will show you how to create your own themes, how to understand some of the code you will see in the existing themes, and how to configure VuFind® to use different themes. 7.2 Using the Theme Generator ----------------------------- -While creating a theme is fairly straightforward (just a matter of creating a directory containing a configuration file and some subdirectories under $VUFIND_HOME/themes), VuFind provides a command-line tool to automatically build a sample theme for you, so you have an easy starting point. Just decide on a theme name (for the example here, we will use localtheme), and run: +While creating a theme is fairly straightforward (just a matter of creating a directory containing a configuration file and some subdirectories under $VUFIND_HOME/themes), VuFind® provides a command-line tool to automatically build a sample theme for you, so you have an easy starting point. Just decide on a theme name (for the example here, we will use localtheme), and run: .. code-block:: console @@ -38,9 +38,9 @@ This will create a new $VUFIND_HOME/theme/localtheme folder containing some exam 7.3 Understanding theme.config.php ---------------------------------- -Every theme in VuFind, including the one you created with the generator in 7.2, will have a theme.config.php file in its root directory. At a bare minimum, this file simply defines a parent theme through the ‘extends’ setting, establishing the aforementioned inheritance mechanism. The file can also be used to define globally-loaded resources like extra Javascript and CSS files, to set a ‘favicon’ (the small graphic that browsers associate with your site in the address bar and bookmarks), or to set up custom view helpers (discussed further in 7.5). +Every theme in VuFind®, including the one you created with the generator in 7.2, will have a theme.config.php file in its root directory. At a bare minimum, this file simply defines a parent theme through the ‘extends’ setting, establishing the aforementioned inheritance mechanism. The file can also be used to define globally-loaded resources like extra Javascript and CSS files, to set a ‘favicon’ (the small graphic that browsers associate with your site in the address bar and bookmarks), or to set up custom view helpers (discussed further in 7.5). -It is worthwhile to take a look at some of the theme.config.php files included with VuFind to gain a better understanding of the available options and to see some examples of how things are formatted. The ‘root’ theme simply defines commonly-used view helpers. Other themes extend this with the necessary Javascript frameworks and CSS styles. +It is worthwhile to take a look at some of the theme.config.php files included with VuFind® to gain a better understanding of the available options and to see some examples of how things are formatted. The ‘root’ theme simply defines commonly-used view helpers. Other themes extend this with the necessary Javascript frameworks and CSS styles. Once you have established your theme, you probably won’t come back and edit theme.config.php very frequently unless you are adding new styles or building custom view helpers. These situations will be discussed in more detail later in the book. @@ -50,12 +50,12 @@ Once you have established your theme, you probably won’t come back and edit th 7.4.1 Basic Technology ______________________ -VuFind’s views are built in native PHP, but they are loaded using the laminas-view component, which sets up some conventions and adds some useful features. PHP already gives you the ability to mix HTML markup with dynamic computer code. The laminas-view component provides a standard way for the business logic code of VuFind to pass variables to the view templates. The job of a template designer is simply to set up a useful presentation of those variables. +VuFind®’s views are built in native PHP, but they are loaded using the laminas-view component, which sets up some conventions and adds some useful features. PHP already gives you the ability to mix HTML markup with dynamic computer code. The laminas-view component provides a standard way for the business logic code of VuFind® to pass variables to the view templates. The job of a template designer is simply to set up a useful presentation of those variables. 7.4.2 Template Locations ________________________ -VuFind’s templates can be found in the templates directory of its themes. When VuFind needs to display a template, it first looks in the currently active theme’s templates directory. If no match is found there, it tries the parent theme’s templates directory. It continues searching from parent to parent until it either finds the template it needs, or it runs out of themes to search. If you need to find a particular template, you can always do the same thing: look at theme.config.php to identify the parent of your theme, and keep tracing your way back until you find where the templates reside. As mentioned earlier, VuFind has some themes that are CSS-only, and others that contain the bulk of the templates. +VuFind®’s templates can be found in the templates directory of its themes. When VuFind® needs to display a template, it first looks in the currently active theme’s templates directory. If no match is found there, it tries the parent theme’s templates directory. It continues searching from parent to parent until it either finds the template it needs, or it runs out of themes to search. If you need to find a particular template, you can always do the same thing: look at theme.config.php to identify the parent of your theme, and keep tracing your way back until you find where the templates reside. As mentioned earlier, VuFind® has some themes that are CSS-only, and others that contain the bulk of the templates. 7.4.3 Template Naming @@ -63,12 +63,12 @@ _____________________ Most of the folders under the templates directory are named after the controllers or plug-ins that they are associated with. Some templates are “partials” – rules for rendering commonly-used chunks of content that may be used in multiple places. For example, the header and footer displayed on every page of the interface are split out into their own header.phtml and footer.phtml files, to make these commonly-customized regions easier to override independently. -It is not always easy to guess exactly which file will contain the part of the interface that you need to override; it is very helpful to use an editing tool with a “search within files” feature when working on VuFind, since it is usually fairly straightforward to find the template you wish to edit by searching for distinctive CSS classes or text labels found within the page you wish to modify. +It is not always easy to guess exactly which file will contain the part of the interface that you need to override; it is very helpful to use an editing tool with a “search within files” feature when working on VuFind®, since it is usually fairly straightforward to find the template you wish to edit by searching for distinctive CSS classes or text labels found within the page you wish to modify. 7.4.4 Reading a PHP Template ____________________________ -The more you understand about both HTML and PHP, the more comfortable you will be working with VuFind’s templates, and the more powerfully you can modify them. However, even if your understanding of these things is fairly limited, you should still be able to recognize patterns and make simple changes like rearranging content, adding labels, etc. +The more you understand about both HTML and PHP, the more comfortable you will be working with VuFind®’s templates, and the more powerfully you can modify them. However, even if your understanding of these things is fairly limited, you should still be able to recognize patterns and make simple changes like rearranging content, adding labels, etc. Most of a template is just plain HTML. However, you may see some :code:`` blocks containing PHP logic, and some :code:`` blocks used for displaying variables inline. For example, you might see something like this: @@ -87,12 +87,12 @@ This is a greatly simplified example, but it demonstrates the basic flavor of te While it is possible to write large amounts of PHP logic directly into template files, this is generally discouraged, as it makes templates harder to read; the focus of a template should be on presentation rather than logic. However, sometimes there is a need to do some complex data processing on a variable before displaying it, or there may be a repetitive task (like rendering a formatted table) that is better done with reusable code than with copy-and-paste. In these situations, a view helper may be useful. -View helpers are a feature of the laminas-view component – they provide a mechanism for hooking up PHP classes full of logic with your view templates in a concise way. The laminas-view component includes a number of useful helpers for common tasks, and VuFind adds many additional specialized options. +View helpers are a feature of the laminas-view component – they provide a mechanism for hooking up PHP classes full of logic with your view templates in a concise way. The laminas-view component includes a number of useful helpers for common tasks, and VuFind® adds many additional specialized options. 7.5.1 Internationalization __________________________ -Two of the most common view helpers you will see in VuFind’s templates are :code:`$this->translate()` and :code:`$this->transEsc()`. These are part of VuFInd’s internationalization system – they make it possible for users to experience the VuFind user interface in a variety of languages. Any text that is passed to these functions is looked up in the language files found under $VUFIND_HOME/languages, and the equivalent text from the user’s selected language is displayed in place of the input. Understanding this is important for several reasons. First of all, if you are searching through templates trying to find a particular piece of text and cannot find it, it may be because the text actually comes from one of the language files, and is represented in the templates as an abbreviated token. Secondly, if you operate a VuFind instance that supports multiple languages, you will need to use these view helpers and populate custom language files (under $VUFIND_LOCAL_DIR/languages) to ensure a correct experience for users of all languages. Finally, when reading templates, it’s important to understand what the translate helpers are doing. +Two of the most common view helpers you will see in VuFind®’s templates are :code:`$this->translate()` and :code:`$this->transEsc()`. These are part of VuFInd’s internationalization system – they make it possible for users to experience the VuFind® user interface in a variety of languages. Any text that is passed to these functions is looked up in the language files found under $VUFIND_HOME/languages, and the equivalent text from the user’s selected language is displayed in place of the input. Understanding this is important for several reasons. First of all, if you are searching through templates trying to find a particular piece of text and cannot find it, it may be because the text actually comes from one of the language files, and is represented in the templates as an abbreviated token. Secondly, if you operate a VuFind® instance that supports multiple languages, you will need to use these view helpers and populate custom language files (under $VUFIND_LOCAL_DIR/languages) to ensure a correct experience for users of all languages. Finally, when reading templates, it’s important to understand what the translate helpers are doing. 7.5.2 Escaping ______________ @@ -103,36 +103,36 @@ It is also important to understand the difference between :code:`$this->translat 7.6 Understanding Layouts -------------------------- -One of VuFind’s most important templates can be found in layout/layout.phtml under the templates directory. This layout template is used to provide the overall structure of every page in VuFind’s interface. The other templates are used to fill in the “content” block at the center of this template. If you need to change the overall structure of pages displayed on your VuFind instance, this is the template you will want to customize. It is also important to understand that the layout template is always rendered last. VuFind first processes the inner content template, then inserts it into the layout. The :code:`$this->layout()` view helper can be used to share information between the layout and the inner templates, but of course this sharing can only happen in one direction: values set inside the layout helper by inner templates can be accessed by the layout, but the reverse is not possible. By the time the layout sets a value, the inner template has already been fully rendered and is no longer in a position to receive that information. +One of VuFind®’s most important templates can be found in layout/layout.phtml under the templates directory. This layout template is used to provide the overall structure of every page in VuFind®’s interface. The other templates are used to fill in the “content” block at the center of this template. If you need to change the overall structure of pages displayed on your VuFind® instance, this is the template you will want to customize. It is also important to understand that the layout template is always rendered last. VuFind® first processes the inner content template, then inserts it into the layout. The :code:`$this->layout()` view helper can be used to share information between the layout and the inner templates, but of course this sharing can only happen in one direction: values set inside the layout helper by inner templates can be accessed by the layout, but the reverse is not possible. By the time the layout sets a value, the inner template has already been fully rendered and is no longer in a position to receive that information. 7.7 Configuring Multiple Themes ------------------------------- -There are several settings in the [Site] section of config.ini that you can adjust to control how VuFind loads themes. +There are several settings in the [Site] section of config.ini that you can adjust to control how VuFind® loads themes. -Most obvious is the “theme” setting: this is the theme that will be loaded by default when a user accesses VuFind for the first time. +Most obvious is the “theme” setting: this is the theme that will be loaded by default when a user accesses VuFind® for the first time. -There is also a “mobile_theme” setting which can be used to load a different theme when a mobile device is detected. This used to be more important when it was popular to display completely different interfaces on mobile vs. desktop devices; now that “responsive design” is the norm, there is very little reason to use this option, and VuFind no longer includes a mobile-specific example theme. +There is also a “mobile_theme” setting which can be used to load a different theme when a mobile device is detected. This used to be more important when it was popular to display completely different interfaces on mobile vs. desktop devices; now that “responsive design” is the norm, there is very little reason to use this option, and VuFind® no longer includes a mobile-specific example theme. -The “alternate_themes” setting allows you to create a list of themes that can be accessed by passing a ?ui= parameter on the end of your VuFind URL. This is a comma-separated list of themes, which are represented as colon-separated pairs. In each of these pairs, the first value is the text that can be passed in as the ui parameter, and the second value is the name of the actual theme to load when that value is passed in. For example: +The “alternate_themes” setting allows you to create a list of themes that can be accessed by passing a ?ui= parameter on the end of your VuFind® URL. This is a comma-separated list of themes, which are represented as colon-separated pairs. In each of these pairs, the first value is the text that can be passed in as the ui parameter, and the second value is the name of the actual theme to load when that value is passed in. For example: :code:`alternate_themes=my1:MyTheme1,my2:MyTheme2` -would load MyTheme1 if a user added ?ui=my1 to the default VuFind URL, and would load MyTheme2 if they instead passed ?ui=my2. +would load MyTheme1 if a user added ?ui=my1 to the default VuFind® URL, and would load MyTheme2 if they instead passed ?ui=my2. -Finally, the “selectable_themes” setting creates a drop-down list that allows users to switch back and forth between themes. It is also a comma-separated list of colon-separated pairs. These pairs consist of the shorthand name for a theme (i.e. the first part of one of the alternative_themes pairs, or else “standard” for the default theme, or “mobile” for the mobile-specific theme). The order of the list controls the order of the drop-down displayed in VuFind. +Finally, the “selectable_themes” setting creates a drop-down list that allows users to switch back and forth between themes. It is also a comma-separated list of colon-separated pairs. These pairs consist of the shorthand name for a theme (i.e. the first part of one of the alternative_themes pairs, or else “standard” for the default theme, or “mobile” for the mobile-specific theme). The order of the list controls the order of the drop-down displayed in VuFind®. -Most VuFind administrators will only need to provide a single theme, but if you want to offer choice to your users (for example, “light” and “dark” themes for users with different vision preferences), setting up alternate_themes and selectable_themes can be helpful. It may also be useful to set up alternate_themes without selectable_themes if you wish to provide access to a different look and feel without advertising it to most users (for example, if you want to provide beta or testing functionality that is only available to a limited audience). +Most VuFind® administrators will only need to provide a single theme, but if you want to offer choice to your users (for example, “light” and “dark” themes for users with different vision preferences), setting up alternate_themes and selectable_themes can be helpful. It may also be useful to set up alternate_themes without selectable_themes if you wish to provide access to a different look and feel without advertising it to most users (for example, if you want to provide beta or testing functionality that is only available to a limited audience). Additional Resources -------------------- -A video covering many of the topics from this chapter is available through the VuFind website (https://vufind.org/wiki/videos:creating_themes). Further information can be found on the User Interface Customization wiki page (https://vufind.org/wiki/development:architecture:user_interface). +A video covering many of the topics from this chapter is available through the VuFind® website (https://vufind.org/wiki/videos:creating_themes). Further information can be found on the User Interface Customization wiki page (https://vufind.org/wiki/development:architecture:user_interface). Summary ------- -VuFind themes allow you to put all of your presentation-related resources in a single place. VuFind’s “theme inheritance” makes it possible to make a few small changes while inheriting most of the features of a pre-built core theme. Tools are available to automate the creation of new themes. With an understanding of a few configuration settings, HTML and PHP, it is possible to take control of VuFind’s look and feel to meet your local needs. +VuFind® themes allow you to put all of your presentation-related resources in a single place. VuFind®’s “theme inheritance” makes it possible to make a few small changes while inheriting most of the features of a pre-built core theme. Tools are available to automate the creation of new themes. With an understanding of a few configuration settings, HTML and PHP, it is possible to take control of VuFind®’s look and feel to meet your local needs. Review Questions diff --git a/source/08-styling_vufind.rst b/source/08-styling_vufind.rst index 31123e1..15ede56 100644 --- a/source/08-styling_vufind.rst +++ b/source/08-styling_vufind.rst @@ -1,6 +1,6 @@ -######################### -Chapter 8. Styling VuFind -######################### +########################## +Chapter 8. Styling VuFind® +########################## Learning Objectives @@ -8,19 +8,19 @@ Learning Objectives After reading this chapter, you will understand: -• How to apply custom styles to your VuFind theme. -• How to take advantage of CSS frameworks in VuFind. -• How to use a CSS pre-processor with VuFind. +• How to apply custom styles to your VuFind® theme. +• How to take advantage of CSS frameworks in VuFind®. +• How to use a CSS pre-processor with VuFind®. 8.1 Understanding CSS Frameworks -------------------------------- -Since its very earliest releases, VuFind has relied on CSS frameworks to provide a unified look and feel and to simplify some common layout tasks. As of this writing (when VuFind 7.0 was the most recent release), the Bootstrap framework (https://getbootstrap.com/) is used as the foundation for VuFind’s default themes; in earlier releases, Blueprint (http://www.blueprintcss.org/) and YUI (https://yuilibrary.com/) were used. Regardless of which framework is used, its purpose is the same: to establish some core CSS classes that enable useful functionality like responsive design, grid layout, etc. +Since its very earliest releases, VuFind® has relied on CSS frameworks to provide a unified look and feel and to simplify some common layout tasks. As of this writing (when VuFind® 7.0 was the most recent release), the Bootstrap framework (https://getbootstrap.com/) is used as the foundation for VuFind®’s default themes; in earlier releases, Blueprint (http://www.blueprintcss.org/) and YUI (https://yuilibrary.com/) were used. Regardless of which framework is used, its purpose is the same: to establish some core CSS classes that enable useful functionality like responsive design, grid layout, etc. -Going in depth on any particular framework is beyond the scope of this book; however, understanding the framework you are working with will help you to read and customize the templates in your theme. It should be easy to find a primer for any major framework using the search engine of your choice; just be sure to read about the same version of the framework that VuFind is using, since some frameworks can introduce significant changes when they introduce major new versions. (For example, Bootstrap 3 and Bootstrap 4 are quite different from one another; VuFind still uses Bootstrap 3 as of this writing, but it is possible that Bootstrap 4 may be adopted in the future). +Going in depth on any particular framework is beyond the scope of this book; however, understanding the framework you are working with will help you to read and customize the templates in your theme. It should be easy to find a primer for any major framework using the search engine of your choice; just be sure to read about the same version of the framework that VuFind® is using, since some frameworks can introduce significant changes when they introduce major new versions. (For example, Bootstrap 3 and Bootstrap 4 are quite different from one another; VuFind® still uses Bootstrap 3 as of this writing, but it is possible that Bootstrap 4 may be adopted in the future). -If you are not sure what framework your VuFind theme is using, the theme name (or the name of a parent theme) may help to clarify this. (For example, VuFind’s bootstrap3 theme is the basic implementation using the Bootstrap 3 framework; there are some more heavily styled sub-themes of bootstrap3, such as bootprint3 and sandal, but if you examine the theme.config.php file in either of these themes, you will see that it extends bootstrap3). +If you are not sure what framework your VuFind® theme is using, the theme name (or the name of a parent theme) may help to clarify this. (For example, VuFind®’s bootstrap3 theme is the basic implementation using the Bootstrap 3 framework; there are some more heavily styled sub-themes of bootstrap3, such as bootprint3 and sandal, but if you examine the theme.config.php file in either of these themes, you will see that it extends bootstrap3). 8.2 Understanding CSS Preprocessors ----------------------------------- @@ -29,7 +29,7 @@ While CSS is a very powerful language for applying styles to web pages, maintain A CSS Preprocessor defines a new language (usually a superset of CSS itself) which adds new features like nested blocks, variables, etc. The preprocessor also includes a tool (called a compiler) which interprets files in the new language and translates them into regular CSS so that they can be used on the web. Using a preprocessor introduces a small trade-off: you have a more powerful language for styling your website, but you have to remember to run the compiler to build your CSS when you want to deploy changes. -As of this writing, VuFind’s default themes make use of the LESS preprocessor, and VuFind includes tools for compiling LESS into CSS using either pure PHP (which is slower, but has fewer dependencies) or Javascript (which is faster but requires installation of the Node.js environment on your server). All of VuFind’s LESS files are also distributed in SASS/SCSS format (a different, but very similar, preprocessor) to accommodate users who already use that format in their development workflows. Because front-end development is a fast-evolving area, it is possible that additional preprocessors may be supported in the future, and that some of the currently-supported formats may eventually be abandoned. +As of this writing, VuFind®’s default themes make use of the LESS preprocessor, and VuFind® includes tools for compiling LESS into CSS using either pure PHP (which is slower, but has fewer dependencies) or Javascript (which is faster but requires installation of the Node.js environment on your server). All of VuFind®’s LESS files are also distributed in SASS/SCSS format (a different, but very similar, preprocessor) to accommodate users who already use that format in their development workflows. Because front-end development is a fast-evolving area, it is possible that additional preprocessors may be supported in the future, and that some of the currently-supported formats may eventually be abandoned. As with CSS frameworks, going in depth on CSS preprocessors is beyond the scope of this text, but tutorials should be available online for LESS and other popular formats. @@ -41,7 +41,7 @@ If you want to add some custom styles to your local theme, you should consider w 8.3.1 Adding a Custom CSS File _______________________________ -To add a custom CSS file to your theme, you can simply edit your theme’s theme.config.php file, and add a new CSS filename to the ‘css’ element of the configuration. All CSS files listed in this array will be loaded on every page of VuFind’s interface, so this is the best way to make global changes. Because of VuFind’s theme inheritance, it is important to give your CSS file a unique name; if you give it the same name as one of the CSS files in a parent theme, it will completely replace that existing file rather than being added to it. +To add a custom CSS file to your theme, you can simply edit your theme’s theme.config.php file, and add a new CSS filename to the ‘css’ element of the configuration. All CSS files listed in this array will be loaded on every page of VuFind®’s interface, so this is the best way to make global changes. Because of VuFind®’s theme inheritance, it is important to give your CSS file a unique name; if you give it the same name as one of the CSS files in a parent theme, it will completely replace that existing file rather than being added to it. As an example, assume that you have created a custom theme named “localtheme” as described in section 7.2 above, and you want to change the body background color to a pale gray color. @@ -65,9 +65,9 @@ You should add a comma on the end of the ‘extends’ line, and put a ‘css’ 'css' => ['myinstitution.css'] ]; -This tells VuFind to add a CSS file called myinstitution.css to every page of its interface; we chose the name myinstitution.css to avoid any possible naming conflict with the core themes (of course, you could replace “myinstitution” with the actual name of your institution if you wished). You only need to specify the filename itself, not any path information; VuFind will search for this filename in your theme’s css folder, and should it fail to find it, it will also search through all of the parent themes. +This tells VuFind® to add a CSS file called myinstitution.css to every page of its interface; we chose the name myinstitution.css to avoid any possible naming conflict with the core themes (of course, you could replace “myinstitution” with the actual name of your institution if you wished). You only need to specify the filename itself, not any path information; VuFind® will search for this filename in your theme’s css folder, and should it fail to find it, it will also search through all of the parent themes. -In order to ensure that VuFind actually finds something when it does its search, you should also create the expected file by editing $VUFIND_HOME/themes/localtheme/css/myinstitution.css. You can paste in this content: +In order to ensure that VuFind® actually finds something when it does its search, you should also create the expected file by editing $VUFIND_HOME/themes/localtheme/css/myinstitution.css. You can paste in this content: .. code-block:: css @@ -75,14 +75,14 @@ In order to ensure that VuFind actually finds something when it does its search, background-color: #d0d0d8; } -Now if you refresh VuFind in your browser, you should see that the local theme’s default background color has changed. +Now if you refresh VuFind® in your browser, you should see that the local theme’s default background color has changed. 8.3.2 Adding a Custom LESS File _______________________________ -VuFind’s provided themes are set up so that all of the LESS files provided are compiled into a single CSS file called “compiled.css.” This setup makes adding a new LESS file a little bit complicated. Fortunately, the sample theme created by the generate command (see section 7.2) creates some example LESS files for you, providing a helpful foundation for you to build upon. +VuFind®’s provided themes are set up so that all of the LESS files provided are compiled into a single CSS file called “compiled.css.” This setup makes adding a new LESS file a little bit complicated. Fortunately, the sample theme created by the generate command (see section 7.2) creates some example LESS files for you, providing a helpful foundation for you to build upon. -If you look in $VUFIND_HOME/themes/localtheme/less after generating the theme, you will see three files: compiled.less, which is the top-level file that VuFind will use to compile the LESS into CSS, based on configuration inherited from a parent theme. All this file does is include custom.less, which is the place where you can put your own custom styles. +If you look in $VUFIND_HOME/themes/localtheme/less after generating the theme, you will see three files: compiled.less, which is the top-level file that VuFind® will use to compile the LESS into CSS, based on configuration inherited from a parent theme. All this file does is include custom.less, which is the place where you can put your own custom styles. If you edit custom.less, you will see that its first line is: @@ -90,9 +90,9 @@ If you edit custom.less, you will see that its first line is: @import “bootstrap”; -This pulls in the default Bootstrap framework styles, which you will need to take advantage of the framework and to make sure that default VuFind templates display correctly. You should leave this line alone. +This pulls in the default Bootstrap framework styles, which you will need to take advantage of the framework and to make sure that default VuFind® templates display correctly. You should leave this line alone. -Everything else in custom.less is an example, and you are free to change or remove it. The provided example shows how to define some variables (like “@active-orange” and “@dark-green”) for internal use, and also how to override some core Bootstrap and VuFind variables (like @brand-primary and @body-bg) to change the way the theme looks without having to build CSS stanzas. There are also some more specific example styles below the variables, and the file ends by demonstrating that you can use @import statements to pull in additional files if you want; the home-page.less file is an example of this capability. +Everything else in custom.less is an example, and you are free to change or remove it. The provided example shows how to define some variables (like “@active-orange” and “@dark-green”) for internal use, and also how to override some core Bootstrap and VuFind® variables (like @brand-primary and @body-bg) to change the way the theme looks without having to build CSS stanzas. There are also some more specific example styles below the variables, and the file ends by demonstrating that you can use @import statements to pull in additional files if you want; the home-page.less file is an example of this capability. If you wanted to implement the same background color change that was used as an example in 8.3.1, you could accomplish it here by editing a single variable and then recompiling the LESS. @@ -144,16 +144,16 @@ Once grunt is installed, you can compile your LESS with: Additional Resources -------------------- -The Bootstrap 3 documentation is available at https://getbootstrap.com/docs/3.3/. You can learn more about LESS at the language’s official website (http://lesscss.org/). VuFind’s use of CSS preprocessing is discussed in more detail on this wiki page: https://vufind.org/wiki/development:architecture:less. +The Bootstrap 3 documentation is available at https://getbootstrap.com/docs/3.3/. You can learn more about LESS at the language’s official website (http://lesscss.org/). VuFind®’s use of CSS preprocessing is discussed in more detail on this wiki page: https://vufind.org/wiki/development:architecture:less. Summary ------- -VuFind’s themes are built using popular CSS frameworks, establishing useful conventions and basic functionality. VuFind also uses CSS preprocessing to work around some of the limitations of CSS when designing its styles. When building your own theme, you can choose to add simple CSS files, or you can do a bit more work to access the full power of preprocessing. +VuFind®’s themes are built using popular CSS frameworks, establishing useful conventions and basic functionality. VuFind® also uses CSS preprocessing to work around some of the limitations of CSS when designing its styles. When building your own theme, you can choose to add simple CSS files, or you can do a bit more work to access the full power of preprocessing. Review Questions ---------------- -1. Why does VuFind use CSS frameworks? -2. Why does VuFind use a CSS preprocessor? +1. Why does VuFind® use CSS frameworks? +2. Why does VuFind® use a CSS preprocessor? 3. What are the advantages and disadvantages of using custom CSS vs. using custom LESS? diff --git a/source/09-customizing_record_views.rst b/source/09-customizing_record_views.rst index 39148d9..729b8f1 100644 --- a/source/09-customizing_record_views.rst +++ b/source/09-customizing_record_views.rst @@ -5,33 +5,33 @@ Chapter 9. Customizing Record Views Learning Objectives ------------------- -• VuFind’s record model, and the purpose of record drivers and the record data formatter. +• VuFind®’s record model, and the purpose of record drivers and the record data formatter. • How to customize the display of existing fields. • How to add new fields to records. 9.1 Understanding Record Drivers -------------------------------- -VuFind is designed to allow searching and display of a wide variety of record types across a wide variety of different systems. Its design strikes a balance between taking advantage of common features, so that the same code can be applied to multiple formats and systems, and allowing differences to be recognized, so that distinct individual characteristics of particular records can be represented and taken advantage of. +VuFind® is designed to allow searching and display of a wide variety of record types across a wide variety of different systems. Its design strikes a balance between taking advantage of common features, so that the same code can be applied to multiple formats and systems, and allowing differences to be recognized, so that distinct individual characteristics of particular records can be represented and taken advantage of. -A key component of this design is the concept of “record drivers.” A record driver is a bundle of PHP code and templates representing a particular metadata format. This code is responsible for translating the specific details of that format into more general concepts that VuFind is familiar with, and the templates are responsible for specifying exactly how VuFind should display records of that type in various areas of its user interface: search results, record views, favorite lists, etc. All of VuFind’s other code interacts with record drivers rather than directly with specific types of records. +A key component of this design is the concept of “record drivers.” A record driver is a bundle of PHP code and templates representing a particular metadata format. This code is responsible for translating the specific details of that format into more general concepts that VuFind® is familiar with, and the templates are responsible for specifying exactly how VuFind® should display records of that type in various areas of its user interface: search results, record views, favorite lists, etc. All of VuFind®’s other code interacts with record drivers rather than directly with specific types of records. -Because all of the format-specific logic resides inside the record drivers, and because all record drivers expose the same basic functionality, VuFind is able to use the same code to present data from a wide range of systems. Because each record driver can provide its own templates, it is possible to display fields and express concepts on a format-by-format basis even if the broader VuFind system has no higher-level awareness of them. These two characteristics (code that conforms to a “least common denominator” and templates that allow expression of very specific format-specific features) contribute significant power and flexibility to VuFind. +Because all of the format-specific logic resides inside the record drivers, and because all record drivers expose the same basic functionality, VuFind® is able to use the same code to present data from a wide range of systems. Because each record driver can provide its own templates, it is possible to display fields and express concepts on a format-by-format basis even if the broader VuFind® system has no higher-level awareness of them. These two characteristics (code that conforms to a “least common denominator” and templates that allow expression of very specific format-specific features) contribute significant power and flexibility to VuFind®. If every record driver had to define all of its own functionality and templates, it would be a lot of work to create new drivers, and the code would contain a lot of duplication. Fortunately, similar to themes, record drivers use inheritance to share functionality, so each driver has a parent driver and then specifies the ways in which it differs from that parent. -Every record driver in VuFind inherits from a top-level class called AbstractBase which defines some of the most low-level, fundamental functionality shared by all record drivers. This is extended by a driver called DefaultRecord, which defines some more specific functionality related to bibliographic records. This in turn is extended by drivers that add more details related to specific platforms like Solr, WorldCat, etc. VuFind has some special behavior related to Solr-based record drivers: when it retrieves a record from Solr, it looks at the record_format field and uses its value to determine a record driver name. For example, if record_format is set to “marc”, it will load the SolrMarc record driver. If it can’t find a driver matching the format value, it will fall back to using “SolrDefault,” which is the parent of most of the other Solr-based drivers. This allows a single Solr index to contain a variety of data formats while allowing VuFind to handle each of those formats differently if necessary. +Every record driver in VuFind® inherits from a top-level class called AbstractBase which defines some of the most low-level, fundamental functionality shared by all record drivers. This is extended by a driver called DefaultRecord, which defines some more specific functionality related to bibliographic records. This in turn is extended by drivers that add more details related to specific platforms like Solr, WorldCat, etc. VuFind® has some special behavior related to Solr-based record drivers: when it retrieves a record from Solr, it looks at the record_format field and uses its value to determine a record driver name. For example, if record_format is set to “marc”, it will load the SolrMarc record driver. If it can’t find a driver matching the format value, it will fall back to using “SolrDefault,” which is the parent of most of the other Solr-based drivers. This allows a single Solr index to contain a variety of data formats while allowing VuFind® to handle each of those formats differently if necessary. -Inside your theme’s templates directory (assuming you are looking at a theme that contains templates), there is a RecordDrivers directory. This contains subdirectories that correspond to the names of record drivers. When VuFind needs to display record-related content, it will search this directory for a template matching the relevant record driver. If no match is found, the parent record driver’s directory will be searched, and so on. In the default VuFind themes, the majority of templates are found in the DefaultRecord driver’s folder, with only a few special overrides being defined for more specific drivers. When record driver templates are rendered, VuFind automatically sets them up with a property called $this->driver, which points to the actual record driver object, making all of its data readily available for display. +Inside your theme’s templates directory (assuming you are looking at a theme that contains templates), there is a RecordDrivers directory. This contains subdirectories that correspond to the names of record drivers. When VuFind® needs to display record-related content, it will search this directory for a template matching the relevant record driver. If no match is found, the parent record driver’s directory will be searched, and so on. In the default VuFind® themes, the majority of templates are found in the DefaultRecord driver’s folder, with only a few special overrides being defined for more specific drivers. When record driver templates are rendered, VuFind® automatically sets them up with a property called $this->driver, which points to the actual record driver object, making all of its data readily available for display. -Many VuFind users never need to create their own record drivers, and many customizations can be accomplished by modifying or extending existing drivers. However, understanding the role of record drivers in VuFind’s design will make it easier to understand the platform as a whole. Additionally, VuFind users with more complex use cases (like an index of records representing substantially different metadata formats or material types) can leverage this system to create a powerful and specialized user experience. +Many VuFind® users never need to create their own record drivers, and many customizations can be accomplished by modifying or extending existing drivers. However, understanding the role of record drivers in VuFind®’s design will make it easier to understand the platform as a whole. Additionally, VuFind® users with more complex use cases (like an index of records representing substantially different metadata formats or material types) can leverage this system to create a powerful and specialized user experience. 9.2 Understanding the Record Data Formatter ------------------------------------------- -Prior to release 4.0, many of VuFind’s record driver templates were long and repetitive, building large tables of data extracted from metadata records and Solr. This had the advantage of simplicity, but it was somewhat difficult to read, maintain and customize. It was hard to quickly identify which fields were being displayed in a particular template, and making local customizations required duplicating large amounts of template code, which in turn led to laborious reconciliation work when core templates changed during upgrades. +Prior to release 4.0, many of VuFind®’s record driver templates were long and repetitive, building large tables of data extracted from metadata records and Solr. This had the advantage of simplicity, but it was somewhat difficult to read, maintain and customize. It was hard to quickly identify which fields were being displayed in a particular template, and making local customizations required duplicating large amounts of template code, which in turn led to laborious reconciliation work when core templates changed during upgrades. -To address these issues, VuFind 4.0 introduced a new view helper called the RecordDataFormatter, whose job is to extract a collection of data from a record driver following a configurable set of rules. This change does make VuFind’s learning curve a bit steeper, since making the most of the RecordDataFormatter’s configuration requires some understanding of PHP, and there are quite a few configuration options to navigate. However, the learning cost is counterbalanced by a solution which resolves the template length, readability and maintenance challenges of the old approach. Additionally, for users who do not like the RecordDataFormatter, there is no mandate to use it; it is a view helper used in VuFind’s default templates, but there is nothing preventing you from building a custom template using the older, simpler approach if you prefer having total control over how things are retrieved and laid out. +To address these issues, VuFind® 4.0 introduced a new view helper called the RecordDataFormatter, whose job is to extract a collection of data from a record driver following a configurable set of rules. This change does make VuFind®’s learning curve a bit steeper, since making the most of the RecordDataFormatter’s configuration requires some understanding of PHP, and there are quite a few configuration options to navigate. However, the learning cost is counterbalanced by a solution which resolves the template length, readability and maintenance challenges of the old approach. Additionally, for users who do not like the RecordDataFormatter, there is no mandate to use it; it is a view helper used in VuFind®’s default templates, but there is nothing preventing you from building a custom template using the older, simpler approach if you prefer having total control over how things are retrieved and laid out. To see how the RecordDataFormatter is configured, you can look at the factory code which builds the view helper, which is found in $VUFIND_HOME/module/VuFind/src/VuFind/View/Helper/Root/RecordDataFormatterFactory.php. If you are not familiar with PHP, this file will be quite intimidating at first glance, but once you recognize some basic patterns, most of it should make sense even if you do not fully understand the programming language behind it. At the top of the file, there is a method named __invoke(), which does the high-level work of building the view helper. You will notice several “setDefaults” lines, which establish default rules for which fields should be retrieved on particular templates. For example: @@ -55,9 +55,9 @@ The *setLine* method offers the absolute simplest example. For example, take a l This simply says “retrieve any values from the record driver’s *getPhysicalDescriptions()* method, and display them with a label of ‘Physical Description:’.” -This simple case, taking only two parameters (label and method) takes advantage of the SpecBuilder’s defaults. The *setLine()* method can accept up to four parameters, with the third (render type) and fourth (options array) opening up a lot of advanced features and behaviors. You should refer to the VuFind wiki (see Additional Resources below) for more details; new options are added from time to time, and the documentation will give you the most up-to-date possibilities. +This simple case, taking only two parameters (label and method) takes advantage of the SpecBuilder’s defaults. The *setLine()* method can accept up to four parameters, with the third (render type) and fourth (options array) opening up a lot of advanced features and behaviors. You should refer to the VuFind® wiki (see Additional Resources below) for more details; new options are added from time to time, and the documentation will give you the most up-to-date possibilities. -The other two methods(*setTemplateLine()* and *setMultiLine()*) are actually wrappers around *setLine()* which make it more convenient to set up some commonly-used advanced configurations. The *setTemplateLine()* method is by far the most commonly-used option; this retrieves data from a record driver method, but instead of displaying it “raw,” it instead passes it to its own record driver template for additional formatting. This is useful for data fields that need to be linked or labeled in special ways. This method is used to display many of VuFind’s default fields, which also means that if you want to change the way those fields are displayed, you can simply override the relevant template in your custom theme without having to touch the RecordDataFormatter configuration. Here is an example: +The other two methods(*setTemplateLine()* and *setMultiLine()*) are actually wrappers around *setLine()* which make it more convenient to set up some commonly-used advanced configurations. The *setTemplateLine()* method is by far the most commonly-used option; this retrieves data from a record driver method, but instead of displaying it “raw,” it instead passes it to its own record driver template for additional formatting. This is useful for data fields that need to be linked or labeled in special ways. This method is used to display many of VuFind®’s default fields, which also means that if you want to change the way those fields are displayed, you can simply override the relevant template in your custom theme without having to touch the RecordDataFormatter configuration. Here is an example: .. code-block:: php @@ -70,7 +70,7 @@ The *setMultiLine()* method is only needed for some rare situations where a sing 9.3 Example: Adding a Field --------------------------- -This has been the most technical chapter of this guide so far, but even if you do not fully understand all of the underlying technology being discussed, you can still take advantage of the software’s power and flexibility. VuFind includes several tools for automatically generating code and configurations, so once you understand some common patterns, you can “fill in the blanks” to accomplish important customizations. This section will guide you step by step through the process of indexing and displaying a local custom field. +This has been the most technical chapter of this guide so far, but even if you do not fully understand all of the underlying technology being discussed, you can still take advantage of the software’s power and flexibility. VuFind® includes several tools for automatically generating code and configurations, so once you understand some common patterns, you can “fill in the blanks” to accomplish important customizations. This section will guide you step by step through the process of indexing and displaying a local custom field. For the purposes of this example, we will assume that your records use the MARC local notes field 597 subfield a to store donor notes, and that you would like to show them as part of your core metadata with a label of “Donor:”. This is a completely fictitious example; the MARC 59x fields are reserved for local use, and every institution may use them in different ways. If you wish to follow along with this example, you can either create some MARC records with fake data in 597, or you can substitute a different field number in the example to pull data from a field that does exist in your records. @@ -79,7 +79,7 @@ The process of setting up a new field requires only three steps. 9.3.1 Step 1: Index the Data ____________________________ -The easiest way to display a new field is to store that data in a field of the Solr index. While VuFind does have the ability to retrieve details from the raw MARC records stored in its index (using special utility methods included in the SolrMarc record driver), pulling the data out to its own field makes it easier to search and facet using that data, and it provides better uniformity if you also expect to work with the same kind of data from non-MARC sources. +The easiest way to display a new field is to store that data in a field of the Solr index. While VuFind® does have the ability to retrieve details from the raw MARC records stored in its index (using special utility methods included in the SolrMarc record driver), pulling the data out to its own field makes it easier to search and facet using that data, and it provides better uniformity if you also expect to work with the same kind of data from non-MARC sources. To index the new field, you should: @@ -88,16 +88,16 @@ To index the new field, you should: 2. Add this line to the marc_local.properties file: *donor_str_mv = 597a* 3. Reindex all of your records as described in section 3.2. -Note that the “donor_str_mv” field name in the example above takes advantage of the “dynamic field suffixes” configured in VuFind’s default schema. While adding a new field to Solr usually requires an edit to the schema file (see section 5.1.2), it is possible to define fields based on patterns, so that, for example, any field name ending with “_str_mv” is recognized as a multi-valued string field. VuFind’s wiki (https://vufind.org/wiki/development:architecture:solr_index_schema#dynamic_field_suffixes) details all of the available dynamic field suffixes. +Note that the “donor_str_mv” field name in the example above takes advantage of the “dynamic field suffixes” configured in VuFind®’s default schema. While adding a new field to Solr usually requires an edit to the schema file (see section 5.1.2), it is possible to define fields based on patterns, so that, for example, any field name ending with “_str_mv” is recognized as a multi-valued string field. VuFind®’s wiki (https://vufind.org/wiki/development:architecture:solr_index_schema#dynamic_field_suffixes) details all of the available dynamic field suffixes. 9.3.2 Step 2: Create a Custom Record Driver ___________________________________________ -Now that your index contains data in the donor_str_mv field, you need to tell VuFind how to read the new field. This requires the addition of a new record driver method. Since we are working with MARC records in this example, the easiest way to set up this method is to extend the SolrDefault record driver in a local code module. +Now that your index contains data in the donor_str_mv field, you need to tell VuFind® how to read the new field. This requires the addition of a new record driver method. Since we are working with MARC records in this example, the easiest way to set up this method is to extend the SolrDefault record driver in a local code module. -Depending on how you installed VuFind, you may or may not already have a local module set up; if you do not, or if you are not sure, you can read ahead to section 16.3.3 for more details. For the purposes of this example, we will assume that you have a local module called MyModule. You should substitute “MyModule” with your actual module name in all of the subsequent example commands. +Depending on how you installed VuFind®, you may or may not already have a local module set up; if you do not, or if you are not sure, you can read ahead to section 16.3.3 for more details. For the purposes of this example, we will assume that you have a local module called MyModule. You should substitute “MyModule” with your actual module name in all of the subsequent example commands. -As discussed further in section 17.2.1, VuFind contains a code generator tool called “extendclass” which can be used to override any core service or plug-in with code in your local module. This saves a lot of time setting up files and configurations. To create the local custom record driver, these commands can be run: +As discussed further in section 17.2.1, VuFind® contains a code generator tool called “extendclass” which can be used to override any core service or plug-in with code in your local module. This saves a lot of time setting up files and configurations. To create the local custom record driver, these commands can be run: .. code-block:: console @@ -186,7 +186,7 @@ You should fill in the file with this code: } } -This code takes advantage of PHP inheritance – it calls the core code’s getDefaultCoreSpecs() method to get default configurations, then adds an additional line to display the donors. This way, even if the core code changes in a future VuFind release to add more fields, we can benefit from those improvements while still adding our additional local field. +This code takes advantage of PHP inheritance – it calls the core code’s getDefaultCoreSpecs() method to get default configurations, then adds an additional line to display the donors. This way, even if the core code changes in a future VuFind® release to add more fields, we can benefit from those improvements while still adding our additional local field. By passing additional options, and by manipulating the array inherited from the parent code, it is also possible to change the order of fields, remove unwanted fields, etc. This example is designed to be as simple as possible; see the links under “Additional Resources” for some more advanced discussion and examples. @@ -206,22 +206,22 @@ In any case, now that the factory is built, the last step is to register it in o (The helpers section is the important part for the purposes of this example; if you have made other customizations, be sure to reconcile this addition with whatever existing configuration you have). -With all of these changes in place, you should now be able to access a record in the VuFind web interface and see the “Donors” display (assuming the record has an underlying 597 field). If this does not work, make sure that you replaced all instances of “MyModule” and “localtheme” in code and commands with the appropriate equivalents if your module or theme has a different name. Also, if you had to create a new local code module, make sure that you remembered to restart Apache to load the updated configuration, and double-check that you cleared your configuration cache as described at the end of step 2. +With all of these changes in place, you should now be able to access a record in the VuFind® web interface and see the “Donors” display (assuming the record has an underlying 597 field). If this does not work, make sure that you replaced all instances of “MyModule” and “localtheme” in code and commands with the appropriate equivalents if your module or theme has a different name. Also, if you had to create a new local code module, make sure that you remembered to restart Apache to load the updated configuration, and double-check that you cleared your configuration cache as described at the end of step 2. Additional Resources -------------------- -VuFind’s wiki contains several pages that support and expand upon the information discussed in this chapter: the Record Driver page (https://vufind.org/wiki/development:plugins:record_drivers) and the RecordDataFormatter reference page (https://vufind.org/wiki/development:architecture:record_data_formatter) are of particular interest. +VuFind®’s wiki contains several pages that support and expand upon the information discussed in this chapter: the Record Driver page (https://vufind.org/wiki/development:plugins:record_drivers) and the RecordDataFormatter reference page (https://vufind.org/wiki/development:architecture:record_data_formatter) are of particular interest. Summary ------- -VuFind’s record driver system isolates format-specific details in a single place, allowing VuFind’s other code to be written in a more generic, reusable way. The RecordDataFormatter provides a powerful way to control and customize the way VuFind displays records. Once you understand these things, you can take control of VuFind’s presentation of data, and you can add local customizations in a few straightforward steps with the help of VuFind’s built-in code generation tools. +VuFind®’s record driver system isolates format-specific details in a single place, allowing VuFind®’s other code to be written in a more generic, reusable way. The RecordDataFormatter provides a powerful way to control and customize the way VuFind® displays records. Once you understand these things, you can take control of VuFind®’s presentation of data, and you can add local customizations in a few straightforward steps with the help of VuFind®’s built-in code generation tools. Review Questions ---------------- -1. What is the responsibility of a record driver in VuFind? -2. What is special about the way VuFind loads record drivers representing Solr records? +1. What is the responsibility of a record driver in VuFind®? +2. What is special about the way VuFind® loads record drivers representing Solr records? 3. What problems inspired the creation of the RecordDataFormatter? 4. What are the three main methods of the RecordDataFormatter\SpecBuilder, and how do they differ from one another? diff --git a/source/10-content_harvesting.rst b/source/10-content_harvesting.rst index 69ea16c..71fb472 100644 --- a/source/10-content_harvesting.rst +++ b/source/10-content_harvesting.rst @@ -7,31 +7,31 @@ Learning Objectives After reading this chapter, you will understand: -• How VuFind uses the OAI-PMH protocol. -• How to load XML-based data into VuFind’s index using XSLT. +• How VuFind® uses the OAI-PMH protocol. +• How to load XML-based data into VuFind®’s index using XSLT. 10.1 Understanding OAI-PMH -------------------------- -OAI-PMH stands for the Open Archives Initiative Protocol for Metadata Harvesting. It provides a relatively simple set of rules for exchanging metadata between computer systems using XML documents. The standard was developed in the early 2000’s, with the current version of the protocol (2.0) being finalized in 2002. While it shows its age in some ways (such as its exclusive reliance on XML formats), it is widely adopted and supported, making it a very useful tool for obtaining records to index into VuFind. +OAI-PMH stands for the Open Archives Initiative Protocol for Metadata Harvesting. It provides a relatively simple set of rules for exchanging metadata between computer systems using XML documents. The standard was developed in the early 2000’s, with the current version of the protocol (2.0) being finalized in 2002. While it shows its age in some ways (such as its exclusive reliance on XML formats), it is widely adopted and supported, making it a very useful tool for obtaining records to index into VuFind®. At its most basic, OAI-PMH allows a client machine (called a “service provider” in OAI-PMH terminology) to download all of the metadata records from a server machine (called a “data provider” by the protocol). In addition to supporting harvesting of full collections, OAI-PMH also allows a client to harvest records incrementally, receiving a collection of records that have been added, deleted or changed since a particular date. Other features include the ability to request specific subsets of a collection (which are defined by the server providing the records – the protocol sets no guidelines for how sets are determined) and the ability to retrieve records in multiple formats (though only Dublin Core, the favored “least common denominator” XML metadata standard, is guaranteed to be universally supported by all data providers). -VuFind can act as an OAI-PMH service provider, because it includes tools for harvesting and indexing records from an OAI-PMH data provider. It also supports the other side of the protocol and includes built-in functionality that allows it to act as an OAI-PMH data provider if you wish to make your records available to others. This chapter discusses VuFind’s harvesting and data provider support in more detail; chapter 11 will talk about what to do with harvested records once you have them. +VuFind® can act as an OAI-PMH service provider, because it includes tools for harvesting and indexing records from an OAI-PMH data provider. It also supports the other side of the protocol and includes built-in functionality that allows it to act as an OAI-PMH data provider if you wish to make your records available to others. This chapter discusses VuFind®’s harvesting and data provider support in more detail; chapter 11 will talk about what to do with harvested records once you have them. -10.2 VuFind’s OAI-PMH Harvester +10.2 VuFind®’s OAI-PMH Harvester ------------------------------- -VuFind’s OAI-PMH harvester is a command line tool which allows you to harvest metadata from an OAI- PMH data provider into a directory of files on your local disk. This directory can then be indexed into VuFind using the processes described in chapter 11. Rules for harvesting can be specified directly on the command line through a variety of options and switches, or they can be stored in the $VUFIND_LOCAL_DIR/harvest/oai.ini file. The oai.ini method is generally more convenient, and that is the approach that will be described here. +VuFind®’s OAI-PMH harvester is a command line tool which allows you to harvest metadata from an OAI- PMH data provider into a directory of files on your local disk. This directory can then be indexed into VuFind® using the processes described in chapter 11. Rules for harvesting can be specified directly on the command line through a variety of options and switches, or they can be stored in the $VUFIND_LOCAL_DIR/harvest/oai.ini file. The oai.ini method is generally more convenient, and that is the approach that will be described here. 10.2.1 Configuring oai.ini and Running the Harvest __________________________________________________ -As with other configuration files in VuFind, you should initialize your harvesting configuration by copying the default example file from $VUFIND_HOME to $VUFIND_LOCAL_DIR: +As with other configuration files in VuFind®, you should initialize your harvesting configuration by copying the default example file from $VUFIND_HOME to $VUFIND_LOCAL_DIR: .. code-block:: console @@ -95,16 +95,16 @@ As noted above, every OAI-PMH server should support the default “oai_dc” met As you can see, this reveals support for not just the standard oai_dc format, but also for a locally- defined metadata format called oai_doaj. -VuFind includes examples for a variety of common formats, but if you harvest a brand new metadata format, you will also be responsible for defining rules for indexing it into VuFind; this is discussed further in chapter 11. +VuFind® includes examples for a variety of common formats, but if you harvest a brand new metadata format, you will also be responsible for defining rules for indexing it into VuFind®; this is discussed further in chapter 11. 10.2.4 Working with Identifiers _______________________________ As discussed in section 3.5.2, it is very important for every record to have its own unique identifier; without IDs, you can’t index things into Solr in a useful way. Some of the metadata formats provided by OAI-PMH servers – especially the richer ones like MARC -- will already contain useful identifiers. However, it is fairly common that records will not contain unique identifiers, or that they will contain multiple identifiers that are not easily differentiated from one another. -Fortunately, VuFind’s OAI-PMH harvester provides a simple solution to this problem. When the OAI- PMH protocol provides records, it sends not just the raw metadata, but also a header above the metadata that includes additional information. This header always includes a unique ID for every record. The harvester only saves the metadata itself, not the header, but several configuration options exist for injecting values from the header into the metadata as custom XML tags. +Fortunately, VuFind®’s OAI-PMH harvester provides a simple solution to this problem. When the OAI- PMH protocol provides records, it sends not just the raw metadata, but also a header above the metadata that includes additional information. This header always includes a unique ID for every record. The harvester only saves the metadata itself, not the header, but several configuration options exist for injecting values from the header into the metadata as custom XML tags. -If you add “injectId = identifier” to your oai.ini configuration section, then unique IDs from the header will be added to an tag inside the top-level tag of your harvested metadata records. The “identifier” tag is used by all of VuFind’s example indexing configurations, but if you wish to use a different tag name for some reason, you can just specify a different value in the configuration. +If you add “injectId = identifier” to your oai.ini configuration section, then unique IDs from the header will be added to an tag inside the top-level tag of your harvested metadata records. The “identifier” tag is used by all of VuFind®’s example indexing configurations, but if you wish to use a different tag name for some reason, you can just specify a different value in the configuration. It is also worth noting that most OAI-PMH record identifiers are quite verbose – for example, “oai:doaj.org/article:311ce1ec3dea42d2a7db0c3de149d865.” It may be desirable to shorten them and/or remove certain characters in order to improve the readability of URLs and avoid other problems (for example, some web servers may require configuration adjustments to support identifiers containing slash characters). Fortunately, there are oai.ini settings to address this situation as well: the idSearch[] and idReplace[] settings can be used in combination with injectId to perform regular expression replacements on IDs before injecting them into metadata. Regular expressions are also briefly discussed in section 5.2.6; they provide a standard language for matching patterns in text, and can be very useful for transformations like this. @@ -131,7 +131,7 @@ The default behavior of the harvest tool is to create a separate XML file on dis The harvest tool provides configuration settings that allow you to group records together into fewer files. If you add “combineRecords = true” to your oai.ini section, each page of records loaded from the server will be stored in a single file, wrapped up inside a tag. If you want to change the name of the wrapping tag, you can use the combineRecordsTag setting to specify a different tag name. -The combineRecords functionality is ideal for harvesting MARC records; the SolrMarc import tool already knows how to deal with tags in MARC-XML, and it will load the files correctly. If you are working with other types of XML, it may be necessary to modify some of VuFind’s provided example import rules to account for multiple records per file; many of them were designed to assume they would only receive one record at a time, though this may be made more flexible in the future. +The combineRecords functionality is ideal for harvesting MARC records; the SolrMarc import tool already knows how to deal with tags in MARC-XML, and it will load the files correctly. If you are working with other types of XML, it may be necessary to modify some of VuFind®’s provided example import rules to account for multiple records per file; many of them were designed to assume they would only receive one record at a time, though this may be made more flexible in the future. 10.2.6 Troubleshooting _______________________ @@ -140,60 +140,60 @@ The harvest tool is designed to be able to resume after a problem, so if there i A common problem with harvesting has to do with invalid data on the remote server. It is a fairly common situation that OAI-PMH servers do not fully validate the XML that they are generating, and sometimes they include incorrectly formatted or illegal characters that cause validation errors for the client. -VuFind’s harvest tool contains some settings that can help resolve persistent problems related to XML validation. If you add “sanitize = true” to your oai.ini section, VuFind will automatically strip out illegal characters. If you set the badXMLLog setting to a filename, VuFind will store more detailed information about problematic XML in this file, which may be helpful for troubleshooting the issue with the content provider. Finally, the sanitizeRegex[] is a repeatable setting which can be used to provide regular expressions defining characters and patterns to remove from incoming XML. This can usually be left at its default value, but if you run into special situations, this provides the ability to customize the cleanup logic. +VuFind®’s harvest tool contains some settings that can help resolve persistent problems related to XML validation. If you add “sanitize = true” to your oai.ini section, VuFind® will automatically strip out illegal characters. If you set the badXMLLog setting to a filename, VuFind® will store more detailed information about problematic XML in this file, which may be helpful for troubleshooting the issue with the content provider. Finally, the sanitizeRegex[] is a repeatable setting which can be used to provide regular expressions defining characters and patterns to remove from incoming XML. This can usually be left at its default value, but if you run into special situations, this provides the ability to customize the cleanup logic. 10.2.7 The Stand-Alone Harvest Tool ____________________________________ -VuFind’s OAI-PMH harvest tool is also available as a separate project; if you ever need to perform a metadata harvest but do not need the full weight of VuFind, it may be useful to download the separate tool, which is available at https://github.com/vufind-org/vufindharvest. The only differences between the stand-alone version and the version found in VuFind are the name of the directory containing the executable PHP code (bin instead of harvest) and the fact that the stand-alone tool does not automatically look for an oai.ini file, since it has no concept of $VUFIND_HOME or $VUFIND_LOCAL_DIR. Instead, you need to use the “--ini=filename.ini” command-line switch to specify your configuration file. +VuFind®’s OAI-PMH harvest tool is also available as a separate project; if you ever need to perform a metadata harvest but do not need the full weight of VuFind®, it may be useful to download the separate tool, which is available at https://github.com/vufind-org/vufindharvest. The only differences between the stand-alone version and the version found in VuFind® are the name of the directory containing the executable PHP code (bin instead of harvest) and the fact that the stand-alone tool does not automatically look for an oai.ini file, since it has no concept of $VUFIND_HOME or $VUFIND_LOCAL_DIR. Instead, you need to use the “--ini=filename.ini” command-line switch to specify your configuration file. 10.3 Open Source OAI-PMH Servers -------------------------------- -Many commonly-used open source tools (including DSpace, Greenstone, Koha and OJS, the Open Journal System) include OAI-PMH data provider capabilities, as do many public repositories of shared open data (such as the Directory of Open Access Journals, DOAJ). VuFind includes sample configurations for harvesting the most popular of these tools, and those configurations can often be easily adapted to support others. This makes VuFind an ideal tool for creating the search “glue” between an ecosystem of open tools. +Many commonly-used open source tools (including DSpace, Greenstone, Koha and OJS, the Open Journal System) include OAI-PMH data provider capabilities, as do many public repositories of shared open data (such as the Directory of Open Access Journals, DOAJ). VuFind® includes sample configurations for harvesting the most popular of these tools, and those configurations can often be easily adapted to support others. This makes VuFind® an ideal tool for creating the search “glue” between an ecosystem of open tools. 10.3.1 DSpace _____________ -The DSpace repository software contains an OAI-PMH data provider server. Depending on your DSpace version, the specific process for enabling and correctly populating the functionality may vary (see the DSpace documentation at https://wiki.lyrasis.org/display/DSDOC6x/OAI for an example). Several metadata formats are supported, and VuFind includes built-in example configurations for indexing both the simple oai_dc metadata as well as the richer DIM format. +The DSpace repository software contains an OAI-PMH data provider server. Depending on your DSpace version, the specific process for enabling and correctly populating the functionality may vary (see the DSpace documentation at https://wiki.lyrasis.org/display/DSDOC6x/OAI for an example). Several metadata formats are supported, and VuFind® includes built-in example configurations for indexing both the simple oai_dc metadata as well as the richer DIM format. 10.3.2 Koha ___________ -The open source Koha Integrated Library System provides a built-in OAI-PMH data provider service, which can be turned on with a configuration setting (see the Koha manual at https://koha- community.org/manual/18.05/en/html/webservices.html#oai-pmh for details). Once activated, you can point VuFind’s harvester at Koha using the marcxml metadataPrefix in order to retrieve records suitable for indexing with SolrMarc as described in chapter 3. Note that you can batch-load harvested MARC records using the harvest/batch-import-marc.sh script, which behaves very similarly to the harvest/batch-import-xsl.sh script described in section 11.3 below. Your import process will run more quickly if you harvest in groups as described in section 10.2.5. +The open source Koha Integrated Library System provides a built-in OAI-PMH data provider service, which can be turned on with a configuration setting (see the Koha manual at https://koha- community.org/manual/18.05/en/html/webservices.html#oai-pmh for details). Once activated, you can point VuFind®’s harvester at Koha using the marcxml metadataPrefix in order to retrieve records suitable for indexing with SolrMarc as described in chapter 3. Note that you can batch-load harvested MARC records using the harvest/batch-import-marc.sh script, which behaves very similarly to the harvest/batch-import-xsl.sh script described in section 11.3 below. Your import process will run more quickly if you harvest in groups as described in section 10.2.5. 10.3.3. OJS ___________ -The OJS (Open Journal System) publishing platform includes built-in OAI-PMH data provider support as well as a metadata plug-in system which makes it possible to add support for custom metadata formats. VuFind includes sample import rules for both the Dublin Core and NLM (National Library of Medicine) formats. +The OJS (Open Journal System) publishing platform includes built-in OAI-PMH data provider support as well as a metadata plug-in system which makes it possible to add support for custom metadata formats. VuFind® includes sample import rules for both the Dublin Core and NLM (National Library of Medicine) formats. -10.4 VuFind’s OAI-PMH Server +10.4 VuFind®’s OAI-PMH Server ____________________________ -In addition to consuming OAI-PMH records, VuFind can also produce them. While VuFind’s OAI-PMH data provider server is turned off by default, it can be activated by uncommenting and filling in the [OAI] section of config.ini. All of the available settings are described by comments in the .ini file; none are required (simply uncommenting the [OAI] section header is enough to turn on the server), but setting an identifier and repository_name are strongly recommended. Other settings exist to give you control over how your OAI-PMH server presents record sets and metadata formats. +In addition to consuming OAI-PMH records, VuFind® can also produce them. While VuFind®’s OAI-PMH data provider server is turned off by default, it can be activated by uncommenting and filling in the [OAI] section of config.ini. All of the available settings are described by comments in the .ini file; none are required (simply uncommenting the [OAI] section header is enough to turn on the server), but setting an identifier and repository_name are strongly recommended. Other settings exist to give you control over how your OAI-PMH server presents record sets and metadata formats. -Once set up, your OAI-PMH server base URL will be your VuFind URL with “/OAI/Server” appended; for example, Villanova University’s instance is https://library.villanova.edu/Find/OAI/Server. If you remove the “/Server” from the end of the page, you will be presented with a helpful form that you can use to test all of the standard OAI-PMH verbs. +Once set up, your OAI-PMH server base URL will be your VuFind® URL with “/OAI/Server” appended; for example, Villanova University’s instance is https://library.villanova.edu/Find/OAI/Server. If you remove the “/Server” from the end of the page, you will be presented with a helpful form that you can use to test all of the standard OAI-PMH verbs. -It is very important to note that VuFind’s OAI-PMH server will only work correctly if you turn on some optional indexing features; these are discussed below. +It is very important to note that VuFind®’s OAI-PMH server will only work correctly if you turn on some optional indexing features; these are discussed below. 10.4.1 Record Change Tracking ______________________________ -Because an OAI-PMH server needs to be able to provide incremental updates showing which records have been added, changed, or deleted, VuFind needs to store some additional information at index time in order to keep track of these details. This functionality is disabled by default, because it makes the indexing process slower; however, that cost is necessary to achieve the benefit of OAI-PMH server functionality (and also some other potentially useful behavior, like properly-sorted RSS feeds and the ability to filter search results by record age). +Because an OAI-PMH server needs to be able to provide incremental updates showing which records have been added, changed, or deleted, VuFind® needs to store some additional information at index time in order to keep track of these details. This functionality is disabled by default, because it makes the indexing process slower; however, that cost is necessary to achieve the benefit of OAI-PMH server functionality (and also some other potentially useful behavior, like properly-sorted RSS feeds and the ability to filter search results by record age). -If you are only indexing MARC records, activating record change tracking is as simple as uncommenting the first_indexed and last_indexed lines in VuFind’s example marc_local.properties file (see section 3.4.3). If you are also indexing XML records, you will need to ensure that the records contain information about modification dates and that your import rules correctly populate the first_indexed and last_indexed fields in Solr. +If you are only indexing MARC records, activating record change tracking is as simple as uncommenting the first_indexed and last_indexed lines in VuFind®’s example marc_local.properties file (see section 3.4.3). If you are also indexing XML records, you will need to ensure that the records contain information about modification dates and that your import rules correctly populate the first_indexed and last_indexed fields in Solr. -For more information about record change tracking, see the relevant page in the VuFind wiki (https://vufind.org/wiki/indexing:tracking_record_changes). +For more information about record change tracking, see the relevant page in the VuFind® wiki (https://vufind.org/wiki/indexing:tracking_record_changes). (https://vufind.org/wiki/indexing:tracking_record_changes). Additional Resources -------------------- -You can read more about OAI-PMH at the protocol’s official website (https://www.openarchives.org/pmh/). VuFind’s OAI-PMH harvest tool has its own project page (https://github.com/vufind-org/vufindharvest). The VuFind wiki also contains notes on OAI-PMH harvesting (https://vufind.org/wiki/indexing:oai-pmh) and server functionality (https://vufind.org/wiki/indexing:tracking_record_changes#oai-pmh_server_functionality). These topics are also covered in video form here: https://vufind.org/wiki/videos:oai- pmh_server_and_harvest_functionality. +You can read more about OAI-PMH at the protocol’s official website (https://www.openarchives.org/pmh/). VuFind®’s OAI-PMH harvest tool has its own project page (https://github.com/vufind-org/vufindharvest). The VuFind® wiki also contains notes on OAI-PMH harvesting (https://vufind.org/wiki/indexing:oai-pmh) and server functionality (https://vufind.org/wiki/indexing:tracking_record_changes#oai-pmh_server_functionality). These topics are also covered in video form here: https://vufind.org/wiki/videos:oai- pmh_server_and_harvest_functionality. Summary ------- -The OAI-PMH protocol provides a common standard for sharing metadata. VuFind can take advantage of the protocol as both a consumer (“service provider”) and a producer (“data provider”) in order to pull together records from multiple systems and share its collection with others. +The OAI-PMH protocol provides a common standard for sharing metadata. VuFind® can take advantage of the protocol as both a consumer (“service provider”) and a producer (“data provider”) in order to pull together records from multiple systems and share its collection with others. Review Questions ---------------- @@ -201,4 +201,4 @@ Review Questions 1. What are the most important features of the OAI-PMH protocol? 2. What is the difference between an OAI-PMH service provider and an OAI-PMH data provider? 3. What are five commonly-used systems that provide OAI-PMH support? -4. What configuration settings are required to allow VuFind to work as an OAI-PMH server? +4. What configuration settings are required to allow VuFind® to work as an OAI-PMH server? diff --git a/source/11-xml_indexing.rst b/source/11-xml_indexing.rst index d7d8e86..abccfa6 100644 --- a/source/11-xml_indexing.rst +++ b/source/11-xml_indexing.rst @@ -15,24 +15,24 @@ After reading this chapter, you will understand: 11.1. Understanding XSLT ------------------------ -XSLT stands for “eXtensible Stylesheet Language Transformations.” It is a declarative programming language used for defining rules to transform one XML document into a different XML document. An XSLT “program” is actually itself an XML document containing rules for selecting elements from a source document and using them to construct a target document. Since many metadata formats are represented using XML formats, and since Solr uses an XML-based format for updating its index, XSLT serves as a useful tool for indexing metadata into VuFind’s Solr index. +XSLT stands for “eXtensible Stylesheet Language Transformations.” It is a declarative programming language used for defining rules to transform one XML document into a different XML document. An XSLT “program” is actually itself an XML document containing rules for selecting elements from a source document and using them to construct a target document. Since many metadata formats are represented using XML formats, and since Solr uses an XML-based format for updating its index, XSLT serves as a useful tool for indexing metadata into VuFind®’s Solr index. -Several versions of the XSLT standard have been developed over the years, with version 3.0 being the most recent as of this writing. However, the PHP language only supports the original 1.0 version of XSLT; thus, by extension, VuFind also uses XSLT 1.0 for its built-in XML indexing tools. This is limiting in some ways, since later versions of XSLT added more richness to the language; however, most of these limitations can be overcome through one of XSLT’s most useful features: extensibility. The PHP XSLT interpreter allows you to register custom PHP functions so that they can be called from within an XSLT sheet. Thus, if a capability is missing from XSLT itself, you can overcome the limitation by delegating to custom PHP code. +Several versions of the XSLT standard have been developed over the years, with version 3.0 being the most recent as of this writing. However, the PHP language only supports the original 1.0 version of XSLT; thus, by extension, VuFind® also uses XSLT 1.0 for its built-in XML indexing tools. This is limiting in some ways, since later versions of XSLT added more richness to the language; however, most of these limitations can be overcome through one of XSLT’s most useful features: extensibility. The PHP XSLT interpreter allows you to register custom PHP functions so that they can be called from within an XSLT sheet. Thus, if a capability is missing from XSLT itself, you can overcome the limitation by delegating to custom PHP code. -Writing your own XSLT sheets and custom PHP functions is beyond the scope of this book; fortunately, for many common systems, you will not have to, since VuFind provides sample indexing configurations for several popular systems (and many of these examples can be adjusted to suit other systems with only minor changes). With an understanding of VuFind’s command line tools and even a rudimentary understanding of how XSLT works, you should be able to accomplish quite a lot. +Writing your own XSLT sheets and custom PHP functions is beyond the scope of this book; fortunately, for many common systems, you will not have to, since VuFind® provides sample indexing configurations for several popular systems (and many of these examples can be adjusted to suit other systems with only minor changes). With an understanding of VuFind®’s command line tools and even a rudimentary understanding of how XSLT works, you should be able to accomplish quite a lot. -If you want to gain a deeper understanding of the XSLT language, numerous helpful books and online articles exist. Just be sure to check which version of the standard is being discussed, since articles about XSLT 2.0 and newer may include some information that will not apply to VuFind’s basic XSLT 1.0 interpreter. Doug Tidwell’s XSLT, published by O’Reilly, is a thorough guidebook and reference; the first edition of the book (published in 2001) covers XSLT 1.0, while the second edition (published in 2009) discusses both 1.0 and 2.0. +If you want to gain a deeper understanding of the XSLT language, numerous helpful books and online articles exist. Just be sure to check which version of the standard is being discussed, since articles about XSLT 2.0 and newer may include some information that will not apply to VuFind®’s basic XSLT 1.0 interpreter. Doug Tidwell’s XSLT, published by O’Reilly, is a thorough guidebook and reference; the first edition of the book (published in 2001) covers XSLT 1.0, while the second edition (published in 2009) discusses both 1.0 and 2.0. -11.2 VuFind’s Command-Line XSLT Indexer +11.2 VuFind®’s Command-Line XSLT Indexer --------------------------------------- -VuFind’s XSLT indexing tool is found at $VUFIND_HOME/import/import-xsl.php. It can be run from the command line like this: +VuFind®’s XSLT indexing tool is found at $VUFIND_HOME/import/import-xsl.php. It can be run from the command line like this: .. code-block:: console php $VUFIND_HOME/import/import-xsl.php -In this command, is the path to an XML file to index, and is the name of an indexing configuration file. For , you only need to specify a filename, not a full path; that filename will be searched for first in $VUFIND_LOCAL_DIR/import, and then in $VUFIND_HOME/import, allowing you to override configuration files through your local configuration directory following VuFind’s usual conventions. +In this command, is the path to an XML file to index, and is the name of an indexing configuration file. For , you only need to specify a filename, not a full path; that filename will be searched for first in $VUFIND_LOCAL_DIR/import, and then in $VUFIND_HOME/import, allowing you to override configuration files through your local configuration directory following VuFind®’s usual conventions. The properties file specifies which XSLT sheet will be used to perform the transformation; example XSLTs are found in $VUFIND_HOME/import/xsl and can be overridden in the equivalent $VUFIND_LOCAL_DIR subdirectory as needed. The file also specifies which custom PHP functions will be made accessible to the XSLT code, and sets up parameters (like your institution name) that will be passed along to the XSLT. @@ -74,13 +74,13 @@ The result might look something like this: 11.3 Batch-Loading XML ---------------------- -While indexing a single record is useful (especially when developing and testing a new set of import rules), it is much more common to want to ingest a batch of records all at once (such as after performing an OAI-PMH harvest as discussed in the previous chapter). Fortunately, VuFind includes a script to automatically ingest all of the XML files in a directory. It is used like this: +While indexing a single record is useful (especially when developing and testing a new set of import rules), it is much more common to want to ingest a batch of records all at once (such as after performing an OAI-PMH harvest as discussed in the previous chapter). Fortunately, VuFind® includes a script to automatically ingest all of the XML files in a directory. It is used like this: .. code-block:: console $VUFIND_HOME/harvest/batch-import-xsl.sh -The parameter is the name of a directory found under either $VUFIND_LOCAL_DIR/harvest or $VUFIND_HOME/harvest (following the usual VuFind pattern of checking the local directory first). The parameter specifies a configuration filename, exactly as described for the single-file importer in section 11.2. +The parameter is the name of a directory found under either $VUFIND_LOCAL_DIR/harvest or $VUFIND_HOME/harvest (following the usual VuFind® pattern of checking the local directory first). The parameter specifies a configuration filename, exactly as described for the single-file importer in section 11.2. When you run the script, it will create a “processed” subdirectory under . It will index XML files from one at a time, moving them into the “processed” subdirectory when they are successfully imported. Any files that fail to load correctly will not be moved, so you can troubleshoot them at the end of the process. If you ever want to re-index your records, you can simply move the files back out of the processed folder and into the main . @@ -89,16 +89,16 @@ If you performed an OAI-PMH harvest, you may also have a number of files in your Additional Resources -------------------- -The XSLT 1.0 standard used by VuFind can be found at https://www.w3.org/TR/xslt-10/. VuFind’s wiki page discussing XML indexing can be found at https://vufind.org/wiki/indexing:xml. A video about XML indexing can be found at https://vufind.org/wiki/videos:indexing_xml_records. +The XSLT 1.0 standard used by VuFind® can be found at https://www.w3.org/TR/xslt-10/. VuFind®’s wiki page discussing XML indexing can be found at https://vufind.org/wiki/indexing:xml. A video about XML indexing can be found at https://vufind.org/wiki/videos:indexing_xml_records. Summary ------- -VuFind includes tools to leverage XSLT 1.0 to index XML records into Solr. Separate configuration files and XSLT definitions can be created for importing different XML formats. A “test-only” mode makes it possible to preview transformations without modifying Solr prematurely. A batch loading script makes it possible to process folders filled with XML files (such as those produced by the OAI-PMH harvest tool discussed in chapter 10). +VuFind® includes tools to leverage XSLT 1.0 to index XML records into Solr. Separate configuration files and XSLT definitions can be created for importing different XML formats. A “test-only” mode makes it possible to preview transformations without modifying Solr prematurely. A batch loading script makes it possible to process folders filled with XML files (such as those produced by the OAI-PMH harvest tool discussed in chapter 10). Review Questions ---------------- -1. What is XSLT and how does VuFind use it? +1. What is XSLT and how does VuFind® use it? 2. How can you see the result of a record’s XSLT transformation without actually indexing that record into Solr? 3. What command is used for batch-loading harvested XML records, and what parameters does it need? diff --git a/source/12-web_crawling_with_vufind.rst b/source/12-web_crawling_with_vufind.rst index c8280ea..96c4aa8 100644 --- a/source/12-web_crawling_with_vufind.rst +++ b/source/12-web_crawling_with_vufind.rst @@ -1,13 +1,13 @@ -#################################### -Chapter 12. Web Crawling with VuFind -#################################### +##################################### +Chapter 12. Web Crawling with VuFind® +##################################### Learning Objectives ------------------- After reading this chapter, you will understand: • The XML sitemap standard. -• VuFind’s full text indexing capabilities. +• VuFind®’s full text indexing capabilities. • Configuration options for website search. @@ -20,14 +20,14 @@ An XML sitemap is a document that lives on a web server – frequently using the Many search engines will try to automatically locate XML sitemaps, but some also provide mechanisms for specifically reporting sitemaps to them; for example, Google’s Webmaster Tools can be used to request that Google crawls your sitemaps. -VuFind can serve as both a producer and consumer of XML sitemaps. This chapter will address both use cases. +VuFind® can serve as both a producer and consumer of XML sitemaps. This chapter will address both use cases. -12.2 Generating Sitemaps in VuFind ----------------------------------- +12.2 Generating Sitemaps in VuFind® +----------------------------------- -As mentioned in section 4.5.3, there is a sitemap.ini file in VuFind’s standard configuration directory which can be used to configure where and how VuFind builds XML sitemaps. VuFind can, for example, build XML sitemaps documenting all of the records in your index, plus a top-level sitemap index that links to the VuFind-generated sitemap as well as additional sitemaps you specify (useful, for example, if you integrate VuFind with another content management system to build your overall website). +As mentioned in section 4.5.3, there is a sitemap.ini file in VuFind®’s standard configuration directory which can be used to configure where and how VuFind® builds XML sitemaps. VuFind® can, for example, build XML sitemaps documenting all of the records in your index, plus a top-level sitemap index that links to the VuFind®-generated sitemap as well as additional sitemaps you specify (useful, for example, if you integrate VuFind® with another content management system to build your overall website). -To create XML sitemaps for your VuFind instance, follow these steps: +To create XML sitemaps for your VuFind® instance, follow these steps: 1.) If you have not already done so, copy $VUFIND_HOME/config/vufind/sitemap.ini to $VUFIND_LOCAL_DIR/config/vufind/ and edit the file to specify your preferences. The detailed comments included in the .ini file document the purpose and meaning of all of the options. @@ -37,17 +37,17 @@ As records are added or removed from your system, you will need to re-run the si Be sure that the user running the sitemap generation process has appropriate permissions to write files into the directory where your sitemaps will be served from. -12.3 VuFind’s Web Crawler -------------------------- +12.3 VuFind®’s Web Crawler +-------------------------- -VuFind is often used in combination with a content management system like WordPress, Drupal or Concrete5; VuFind provides the search capabilities while the CMS provides static content. For example, a library might use VuFind for catalog searching, but have a CMS for providing lists of online resources, guides for researching particular topics, etc. In this kind of scenario, it is useful to be able to make all of the web content searchable through VuFind, so users can perform a full text search across the entire site. To support this use case, VuFind includes a separate Solr core designed for searching web pages. The default tool for populating this index relies on XML sitemaps to retrieve content. Fortunately, most commonly used platforms already support sitemap generation, and the format is so simple that it is possible to build them by hand (for a small number of pages) or develop tools to help build them (if you have some basic experience with script-writing). +VuFind® is often used in combination with a content management system like WordPress, Drupal or Concrete5; VuFind® provides the search capabilities while the CMS provides static content. For example, a library might use VuFind® for catalog searching, but have a CMS for providing lists of online resources, guides for researching particular topics, etc. In this kind of scenario, it is useful to be able to make all of the web content searchable through VuFind®, so users can perform a full text search across the entire site. To support this use case, VuFind® includes a separate Solr core designed for searching web pages. The default tool for populating this index relies on XML sitemaps to retrieve content. Fortunately, most commonly used platforms already support sitemap generation, and the format is so simple that it is possible to build them by hand (for a small number of pages) or develop tools to help build them (if you have some basic experience with script-writing). -Setting up web indexing in VuFind is a two-step process: first, you must configure a tool to extract searchable text from web pages; then, you need to configure VuFind’s web crawling tool. The following two sections discuss these steps in more detail. +Setting up web indexing in VuFind® is a two-step process: first, you must configure a tool to extract searchable text from web pages; then, you need to configure VuFind®’s web crawling tool. The following two sections discuss these steps in more detail. 12.3.1 Setting Up Full Text Indexing ____________________________________ -In order to perform full-text searching of a web page, VuFind needs a way to extract all of the searchable text from the HTML document. Fortunately, several tools exist for extracting text from documents, and VuFind provides built-in support for integrating with two of these: Apache Tika, one of the most popular options, as well as Aperture, an older and lesser-used tool. Most users will want to use Tika, as it is better-maintained and supported. +In order to perform full-text searching of a web page, VuFind® needs a way to extract all of the searchable text from the HTML document. Fortunately, several tools exist for extracting text from documents, and VuFind® provides built-in support for integrating with two of these: Apache Tika, one of the most popular options, as well as Aperture, an older and lesser-used tool. Most users will want to use Tika, as it is better-maintained and supported. To get set up, follow these steps: @@ -56,12 +56,12 @@ To get set up, follow these steps: 3.) Edit the new local copy of fulltext.ini. Uncomment the [General] section header and the “parser=Tika” lines to activate Tika support, and set the “path” setting under the [Tika] section to the full path to your downloaded jar file – e.g. /usr/local/tika/tika-app-1.24.jar. -Now, when VuFind needs to extract full text from a document, it will use Tika to do so. This is useful not only for website indexing, but also for extracting full text from documents referenced in metadata files indexed into your main biblio core. Tika supports many document formats, so it is useful not just for HTML documents but also for Microsoft Word files, PDFs, etc. For example, if you want to index the content of documents referenced in 856 fields in MARC records, you can now use SolrMarc’s custom getFulltext method (see example in $VUFIND_HOME/import/marc_local.properties). Similarly, if you want to index linked content in XML metadata, you can use VuFind’s custom harvestWithParser() method, as demonstrated by the example in $VUFIND_HOME/import/xsl/nlm_ojs.xsl. +Now, when VuFind® needs to extract full text from a document, it will use Tika to do so. This is useful not only for website indexing, but also for extracting full text from documents referenced in metadata files indexed into your main biblio core. Tika supports many document formats, so it is useful not just for HTML documents but also for Microsoft Word files, PDFs, etc. For example, if you want to index the content of documents referenced in 856 fields in MARC records, you can now use SolrMarc’s custom getFulltext method (see example in $VUFIND_HOME/import/marc_local.properties). Similarly, if you want to index linked content in XML metadata, you can use VuFind®’s custom harvestWithParser() method, as demonstrated by the example in $VUFIND_HOME/import/xsl/nlm_ojs.xsl. 12.3.2 Configuring and Running the Web Crawler ______________________________________________ -Once full text indexing is configured, setting up VuFind’s web crawler is quite simple. Copy $VUFIND_HOME/config/vufind/webcrawl.ini to $VUFIND_LOCAL_DIR/config/vufind/ and edit the file. Simply add a “url[] =” line to the [Sitemaps] section for every sitemap URL that you wish to crawl. If your site has a sitemapIndex.xml, you can point to this single file and VuFind’s crawler will follow all of the links within it. You can turn on the “verbose” setting in the [General] section if you want to see more detailed output while the indexing process is running. +Once full text indexing is configured, setting up VuFind®’s web crawler is quite simple. Copy $VUFIND_HOME/config/vufind/webcrawl.ini to $VUFIND_LOCAL_DIR/config/vufind/ and edit the file. Simply add a “url[] =” line to the [Sitemaps] section for every sitemap URL that you wish to crawl. If your site has a sitemapIndex.xml, you can point to this single file and VuFind®’s crawler will follow all of the links within it. You can turn on the “verbose” setting in the [General] section if you want to see more detailed output while the indexing process is running. Once the configuration is set up, you can initiate the crawling process with the command: @@ -72,12 +72,12 @@ Once the configuration is set up, you can initiate the crawling process with the This will index all of the pages in all of the sitemaps referenced in webcrawl.ini. This process can take a long time for large sites, since it has to download every web page in order to index it. When it finishes indexing new pages, it will delete any pages in the index that have ceased to exist since the last time the tool was launched. For this reason, you should never run the web indexer while your site is offline, since it could end up removing useful content from your index. -The webcrawl.php tool operates by applying an XSLT to the downloaded sitemap.xml files; it is actually a specialized version of the XML indexer described in chapter 11. If you need to make changes to the way pages are indexed (for example, to extract the content of a specific tag into a custom index field for faceting purposes), you can override and customize $VUFIND_HOME/import/sitemap.properties and/or $VUFIND_HOME/import/xsl/sitemap.xsl as needed. For an example of this type of customization, see the WordPress section of the article “The Triumph of David: A Case Study in VuFind Customization,” published in Annals of Library and Information Science v. 63, no. 4 and available online here: http://op.niscair.res.in/index.php/ALIS/article/view/14527. +The webcrawl.php tool operates by applying an XSLT to the downloaded sitemap.xml files; it is actually a specialized version of the XML indexer described in chapter 11. If you need to make changes to the way pages are indexed (for example, to extract the content of a specific tag into a custom index field for faceting purposes), you can override and customize $VUFIND_HOME/import/sitemap.properties and/or $VUFIND_HOME/import/xsl/sitemap.xsl as needed. For an example of this type of customization, see the WordPress section of the article “The Triumph of David: A Case Study in VuFind® Customization,” published in Annals of Library and Information Science v. 63, no. 4 and available online here: http://op.niscair.res.in/index.php/ALIS/article/view/14527. -12.4 Accessing and Customizing VuFind’s Web Search --------------------------------------------------- +12.4 Accessing and Customizing VuFind®’s Web Search +--------------------------------------------------- -Once you have finished indexing content, you can search your web index through VuFind’s separate Web search; the URL will be something like http://localhost/vufind/Web, assuming that http://localhost/vufind/ is your VuFind base URL. +Once you have finished indexing content, you can search your web index through VuFind®’s separate Web search; the URL will be something like http://localhost/vufind/Web, assuming that http://localhost/vufind/ is your VuFind® base URL. If you wish to customize the behavior of the web search, there are several files that you can override as needed: @@ -87,21 +87,21 @@ If you wish to customize the behavior of the web search, there are several files - templates/RecordDriver/SolrWeb/result-list.phtml – This template file can be overridden within your theme to change the way individual web results are displayed in the search result list; the VuFind\RecordDriver\SolrWeb class can also be extended to add functionality as needed. For more on customizing record views, see chapter 9. -Unless you are planning on using VuFind exclusively for web searching, you will likely want to make it convenient for users to seamlessly search across both the web index and the main bibliographic record index. See chapter 13 for more on how to combine different types of searches using VuFind. +Unless you are planning on using VuFind® exclusively for web searching, you will likely want to make it convenient for users to seamlessly search across both the web index and the main bibliographic record index. See chapter 13 for more on how to combine different types of searches using VuFind®. Additional Resources -------------------- -Notes on VuFind’s web crawling tools can be found on this wiki page: https://vufind.org/wiki/indexing:websites. A video discussing sitemaps and web crawling can be found here: https://vufind.org/wiki/videos:sitemaps_and_web_indexing. +Notes on VuFind®’s web crawling tools can be found on this wiki page: https://vufind.org/wiki/indexing:websites. A video discussing sitemaps and web crawling can be found here: https://vufind.org/wiki/videos:sitemaps_and_web_indexing. Summary ------- -XML sitemaps provide a useful way to publish lists of web pages for consumption by search engines. VuFind can produce its own sitemaps to make indexed content more visible in search engines, and it can consume external sitemaps to build its own searchable web page index as a complement to its core bibliographic record index. +XML sitemaps provide a useful way to publish lists of web pages for consumption by search engines. VuFind® can produce its own sitemaps to make indexed content more visible in search engines, and it can consume external sitemaps to build its own searchable web page index as a complement to its core bibliographic record index. Review Questions ---------------- 1. What is the difference between sitemap.xml and sitemapIndex.xml? -2. What configuration files do you need to edit in order to set up web indexing in VuFind? -3. What URL is used to perform searches of VuFind’s web index? +2. What configuration files do you need to edit in order to set up web indexing in VuFind®? +3. What URL is used to perform searches of VuFind®’s web index? diff --git a/source/13-combining_different_types_of_search_results.rst b/source/13-combining_different_types_of_search_results.rst index f36454b..45e6170 100644 --- a/source/13-combining_different_types_of_search_results.rst +++ b/source/13-combining_different_types_of_search_results.rst @@ -15,11 +15,11 @@ After reading this chapter, you will understand: 13.1 Understanding combined.ini ------------------------------- -So far in this book, two distinct types of VuFind searching have been introduced: the default bibliographic search and the web search (see chapter 12). Even more options will be discussed in chapter 15. Search engines have trained users to search using a single box, but presenting heterogeneous search results in a single list does not always make sense, and can introduce complex relevance ranking challenges. A popular solution to this problem is the “Bento Box” approach, in which a single search query reveals multiple categories of search results. VuFind supports this type of searching through the combined.ini configuration file. +So far in this book, two distinct types of VuFind® searching have been introduced: the default bibliographic search and the web search (see chapter 12). Even more options will be discussed in chapter 15. Search engines have trained users to search using a single box, but presenting heterogeneous search results in a single list does not always make sense, and can introduce complex relevance ranking challenges. A popular solution to this problem is the “Bento Box” approach, in which a single search query reveals multiple categories of search results. VuFind® supports this type of searching through the combined.ini configuration file. -The combined.ini file controls the behavior of the /Combined page of your VuFind instance (e.g. http://localhost/vufind/Combined). It allows you to search across any number of search backends and display those results on screen together in a pattern that you define. +The combined.ini file controls the behavior of the /Combined page of your VuFind® instance (e.g. http://localhost/vufind/Combined). It allows you to search across any number of search backends and display those results on screen together in a pattern that you define. -As with all of VuFind’s configuration files, the most complete and up-to-date documentation on options and values can be found in the comments contained within the file itself. The configuration file has a handful of reserved sections (Basic_Searches, HomePage, Layout and RecommendationModules); any other sections added to the file must have names that correspond with search backends supported by VuFind (see section 4.5.2 for an introduction to search backends). +As with all of VuFind®’s configuration files, the most complete and up-to-date documentation on options and values can be found in the comments contained within the file itself. The configuration file has a handful of reserved sections (Basic_Searches, HomePage, Layout and RecommendationModules); any other sections added to the file must have names that correspond with search backends supported by VuFind® (see section 4.5.2 for an introduction to search backends). The reserved sections affect global aspects of the combined screen: how to lay out the Bento Boxes, what to display on the home screen before a search is performed, which recommendation modules to display above or below the combined results, etc. The search backend-specific sections contain settings specifying how to display results from those backends: how many results to include in the box, whether or not to load the results through AJAX, how to label everything, etc. @@ -40,7 +40,7 @@ For example, suppose we want to show our standard Solr search results side-by-si more_link = "More Website Results" -VuFind’s combined search feature also includes the ability to create specialized boxes by applying filters to search backends. For example, if you wanted to highlight online materials in your catalog, you could create a special box to show them. To do this, you simply need to add a colon and a differentiating name after the search backend name in the configuration, then populate the filter or hiddenFilter setting. The difference between a “filter” and a “hiddenFilter” is whether or not the user interface will show the filter as an applied facet. If you use “filter” and the user clicks through to the full search results, they will see the filter and can remove it; if you use “hiddenFilter,” the filter will be applied but there will be no control to get rid of it. The “hiddenFilter” control is useful in combination with filtered search tabs (described below in section 13.3), but “filter” will provide a better user experience in other situations. The “online” example could be set up like this: +VuFind®’s combined search feature also includes the ability to create specialized boxes by applying filters to search backends. For example, if you wanted to highlight online materials in your catalog, you could create a special box to show them. To do this, you simply need to add a colon and a differentiating name after the search backend name in the configuration, then populate the filter or hiddenFilter setting. The difference between a “filter” and a “hiddenFilter” is whether or not the user interface will show the filter as an applied facet. If you use “filter” and the user clicks through to the full search results, they will see the filter and can remove it; if you use “hiddenFilter,” the filter will be applied but there will be no control to get rid of it. The “hiddenFilter” control is useful in combination with filtered search tabs (described below in section 13.3), but “filter” will provide a better user experience in other situations. The “online” example could be set up like this: .. code-block:: ini @@ -54,12 +54,12 @@ This example uses a hiddenFilter because it is designed to complement a future s Note that if you copy the default combined.ini file from $VUFIND_HOME/config/vufind/combined.ini, you will have to comment out any example sections representing search backends that you do not use; the simple presence of a backend-related section in the file will cause that backend to be included in combined results. -If you want to make this combined search the default for your entire site, you can edit your $VUFIND_LOCAL_DIR/config/vufind/config.ini file and change the “defaultModule” setting from “Search” (the default, which uses the primary Solr biblio core as the standard search) to “Combined” (which will make the global search box do a combined search by default, and will make the standard home page of VuFind the Combined home page). +If you want to make this combined search the default for your entire site, you can edit your $VUFIND_LOCAL_DIR/config/vufind/config.ini file and change the “defaultModule” setting from “Search” (the default, which uses the primary Solr biblio core as the standard search) to “Combined” (which will make the global search box do a combined search by default, and will make the standard home page of VuFind® the Combined home page). 13.2 Understanding searchbox.ini -------------------------------- -VuFind offers a configuration file to control the behavior of the search box that appears on every page of the application: searchbox.ini. By default, the drop-down menu next to the search box will only include options relevant to the currently-active backend. For example, if you are looking at Solr bibliographic search results, you will see the options configured in searches.ini; if you are looking at web results, you will see the options configured in website.ini. If you introduce combined searching, however, this behavior may no longer be desirable, and it may be preferable to display all of the options in the menu at all times, for a more consistent experience. +VuFind® offers a configuration file to control the behavior of the search box that appears on every page of the application: searchbox.ini. By default, the drop-down menu next to the search box will only include options relevant to the currently-active backend. For example, if you are looking at Solr bibliographic search results, you will see the options configured in searches.ini; if you are looking at web results, you will see the options configured in website.ini. If you introduce combined searching, however, this behavior may no longer be desirable, and it may be preferable to display all of the options in the menu at all times, for a more consistent experience. To turn on combined options in the search box, simply copy $VUFIND_HOME/config/vufind/searchbox.ini into $VUFIND_LOCAL_DIR/config/vufind/, turn on the combinedHandlers option in the [General] section, and populate the [CombinedHandlers] section with the appropriate options. For example: @@ -81,16 +81,16 @@ To turn on combined options in the search box, simply copy $VUFIND_HOME/config/v When editing the [CombinedHandlers] section, it is important to ensure that all of the settings end in brackets ([]) and that every block includes all of the values. Otherwise, the configuration may be interpreted incorrectly. -The “type” of “VuFind” simply indicates that these are internal VuFind search backends; you can also use a type value of “External” if you want to allow the VuFind search box to redirect into a third-party system. (When using “External,” the “target” value is a URL instead of a search backend name). The “label” setting should be self-explanatory, and the “group” setting can be used to group related options together under a heading within the drop-down menu, if you have a large number of options that need to be organized more hierarchically. +The “type” of “VuFind®” simply indicates that these are internal VuFind® search backends; you can also use a type value of “External” if you want to allow the VuFind® search box to redirect into a third-party system. (When using “External,” the “target” value is a URL instead of a search backend name). The “label” setting should be self-explanatory, and the “group” setting can be used to group related options together under a heading within the drop-down menu, if you have a large number of options that need to be organized more hierarchically. As usual, additional options (such as the ability to incorporate alphabetical browse options into the search drop-down menu) are documented through comments in the file. 13.3 Configuring search tabs in config.ini ------------------------------------------ -In addition to the value of searching multiple systems at once and having access to all options through a single drop-down menu, there is one more feature which can help users navigate complex search environments: tabs. While the combined search screen provides a summary of the first page of results from multiple search backends, users will often need to click into a single result set to access deeper results and features like facet controls. Once a user has focused in on a specific result set, it is sometimes useful to have a quick way to switch into a different one. VuFind’s search tab feature offers this functionality. +In addition to the value of searching multiple systems at once and having access to all options through a single drop-down menu, there is one more feature which can help users navigate complex search environments: tabs. While the combined search screen provides a summary of the first page of results from multiple search backends, users will often need to click into a single result set to access deeper results and features like facet controls. Once a user has focused in on a specific result set, it is sometimes useful to have a quick way to switch into a different one. VuFind®’s search tab feature offers this functionality. -When search tabs are enabled, tabs will appear near the search box. When search results are displayed, clicking on a different tab will transfer the current search terms (and, when possible, the search handler) to a different search backend. Tabs provide a quick way to switch between different detailed search result views. The transfer of search handler settings is based on name matching – for example, if you are performing an “Author” search in Solr and you click to a different tab, VuFind will look for a search handler whose description matches “Author;” if no match is found, the default handler will be applied. +When search tabs are enabled, tabs will appear near the search box. When search results are displayed, clicking on a different tab will transfer the current search terms (and, when possible, the search handler) to a different search backend. Tabs provide a quick way to switch between different detailed search result views. The transfer of search handler settings is based on name matching – for example, if you are performing an “Author” search in Solr and you click to a different tab, VuFind® will look for a search handler whose description matches “Author;” if no match is found, the default handler will be applied. Configuration of search tabs takes place in config.ini’s [SearchTabs] section. Simply create a map of search handler names to labels. To continue this chapter’s example of combined bibliographic and website searching, you could use these settings: @@ -100,7 +100,7 @@ Configuration of search tabs takes place in config.ini’s [SearchTabs] section. Solr = Catalog SolrWeb = Website -VuFind’s search tab feature also includes the ability to create specialized tabs by applying filters to search backends, similar to the way combined search Bento Boxes can be filtered. To do this, you simply need to add a colon and a differentiating name after the search backend name in the configuration. Then you need to add an entry to the [SearchTabsFilters] section of the configuration specifying the filter(s) to apply. The “online” tab example from section 13.1 could be set up as a tab like this: +VuFind®’s search tab feature also includes the ability to create specialized tabs by applying filters to search backends, similar to the way combined search Bento Boxes can be filtered. To do this, you simply need to add a colon and a differentiating name after the search backend name in the configuration. Then you need to add an entry to the [SearchTabsFilters] section of the configuration specifying the filter(s) to apply. The “online” tab example from section 13.1 could be set up as a tab like this: .. code-block:: ini @@ -116,12 +116,12 @@ VuFind’s search tab feature also includes the ability to create specialized ta Additional Resources -------------------- -Some additional information on the subjects discussed in this chapter can be found on the “Combining Search Types” page in VuFind’s wiki (https://vufind.org/wiki/configuration:combining_search_types). A video about combining searches can be found here: https://vufind.org/wiki/videos:combining_search_types. +Some additional information on the subjects discussed in this chapter can be found on the “Combining Search Types” page in VuFind®’s wiki (https://vufind.org/wiki/configuration:combining_search_types). A video about combining searches can be found here: https://vufind.org/wiki/videos:combining_search_types. Summary ------- -VuFind supports many different types of searching; by configuring combined.ini, searchbox.ini and config.ini correctly, it is possible to make the user’s experience more convenient and understandable while navigating the available options in your environment. +VuFind® supports many different types of searching; by configuring combined.ini, searchbox.ini and config.ini correctly, it is possible to make the user’s experience more convenient and understandable while navigating the available options in your environment. Review Questions ---------------- diff --git a/source/14-recommendation_modules.rst b/source/14-recommendation_modules.rst index f40f32e..83cc082 100644 --- a/source/14-recommendation_modules.rst +++ b/source/14-recommendation_modules.rst @@ -15,11 +15,11 @@ After reading this chapter, you will understand: 14.1 What is a Recommendation Module? -------------------------------------- -Generally, when a user sees a list of search results, it is beneficial to provide them with some additional information beyond the results themselves: facet lists for limiting results, suggestions about spelling corrections or relevant alternate resources, etc. In VuFind, all of this supplemental information is displayed to the user using “recommendation modules.” +Generally, when a user sees a list of search results, it is beneficial to provide them with some additional information beyond the results themselves: facet lists for limiting results, suggestions about spelling corrections or relevant alternate resources, etc. In VuFind®, all of this supplemental information is displayed to the user using “recommendation modules.” -A recommendation module is a VuFind plug-in, consisting of a PHP class and an associated template, which can examine a user’s search query and/or the search results, then display useful information based on some or all of that input. Recommendation module output can be configured to display above or beside the search results, and different modules can be configured for each search handler. It is also possible to configure VuFind to display different recommendation modules when the user’s search results are empty, since a different set of suggestions may be relevant in that scenario. +A recommendation module is a VuFind® plug-in, consisting of a PHP class and an associated template, which can examine a user’s search query and/or the search results, then display useful information based on some or all of that input. Recommendation module output can be configured to display above or beside the search results, and different modules can be configured for each search handler. It is also possible to configure VuFind® to display different recommendation modules when the user’s search results are empty, since a different set of suggestions may be relevant in that scenario. -VuFind includes a large variety of recommendation modules and a default configuration that makes use of those that are most commonly needed. VuFind’s built-in extensibility also makes it straightforward for developers to build their own custom modules as needed. +VuFind® includes a large variety of recommendation modules and a default configuration that makes use of those that are most commonly needed. VuFind®’s built-in extensibility also makes it straightforward for developers to build their own custom modules as needed. 14.2 Understanding Recommendation Configuration ----------------------------------------------- @@ -34,7 +34,7 @@ This configuration setup offers a balance between ease of use and specificity. Y At each level of the configuration, recommendation modules are configured by providing the name of the module, optionally followed by a set of parameters separated by colons. The meanings of these parameters are detailed in the comments in searches.ini (and some examples will be explained below). -Here are the most relevant configurations from searches.ini as of VuFind 7.0: +Here are the most relevant configurations from searches.ini as of VuFind® 7.0: .. code-block:: ini @@ -78,17 +78,17 @@ Note that for all of the repeating settings (like default_top_recommend[]), it i Chapter 13 discussed an example of combining standard search results with website search results in a variety of ways. With recommendation modules, it is possible to add another layer of combined searching: you can display results from one backend as a sidebar in another result set. For example, you could show the top five web results in a sidebar next to main catalog results, and vice versa. -VuFind provides two recommendation modules that can be used to meet this need: CatalogResults, which displays results from the main Solr biblio core, and WebResults, which displays results from the Solr website core. Both of these modules accept two parameters: the name of the URL parameter containing search terms (which defaults to “lookfor,” which does not need to be changed for this example) and the number of search results to display in the sidebar (which defaults to 5). We just need to turn on WebResults in searches.ini and CatalogResults in website.ini. +VuFind® provides two recommendation modules that can be used to meet this need: CatalogResults, which displays results from the main Solr biblio core, and WebResults, which displays results from the Solr website core. Both of these modules accept two parameters: the name of the URL parameter containing search terms (which defaults to “lookfor,” which does not need to be changed for this example) and the number of search results to display in the sidebar (which defaults to 5). We just need to turn on WebResults in searches.ini and CatalogResults in website.ini. This can be set up by following these steps: 1. Copy searches.ini and website.ini from $VUFIND_HOME/config/vufind/ to $VUFIND_LOCAL_DIR/config/vufind/ if you have not previously customized these files. -2. Edit $VUFIND_LOCAL_DIR/config/vufind/searches.ini, and add *default_side_recommend[] = WebResults* to the [General] section of the file. In a default VuFind configuration, there should be no customizations in the [SideRecommendations] section, but if you have made customizations there, you will want to add WebResults to each of the customized search handlers as well. +2. Edit $VUFIND_LOCAL_DIR/config/vufind/searches.ini, and add *default_side_recommend[] = WebResults* to the [General] section of the file. In a default VuFind® configuration, there should be no customizations in the [SideRecommendations] section, but if you have made customizations there, you will want to add WebResults to each of the customized search handlers as well. -3. Edit $VUFIND_LOCAL_DIR/config/vufind/website.ini and add *default_side_recommend[] = CatalogResults* to the [General] section if it is not already there (in recent VuFind releases, this is already turned on by default). +3. Edit $VUFIND_LOCAL_DIR/config/vufind/website.ini and add *default_side_recommend[] = CatalogResults* to the [General] section if it is not already there (in recent VuFind® releases, this is already turned on by default). -Now VuFind should display brief previews of web results in standard result listings and vice versa. +Now VuFind® should display brief previews of web results in standard result listings and vice versa. If you wanted to display a different number of results in the recommendation boxes (for the sake of example, 3), you could edit the configuration lines to read *default_side_recommend[] = WebResults::3* and *default_side_recommend[] = CatalogResults::3*. The double colon is present because we are leaving the first parameter blank. @@ -97,7 +97,7 @@ If you want to customize the look and feel of the recommendation boxes, each rec 14.4 Example: Displaying Extra Links for Empty Search Results -------------------------------------------------------------- -It is often useful to provide links to specific resources related to a search. For example, you might have a “search tips” page on your website which could provide guidance for users having difficulty with searches. VuFind includes a recommendation module named RecommendLinks which can render such a list. The RecommendLinks helper takes two parameters: the name of an ini file, and the name of a section within that file; it uses these to locate the list of recommendations to display. If no extra details are specified, it will look in the [RecommendLinks] section of searches.ini. Allowing configuration of the location of links means that the same RecommendLinks module can be used in different contexts to display different lists of links. +It is often useful to provide links to specific resources related to a search. For example, you might have a “search tips” page on your website which could provide guidance for users having difficulty with searches. VuFind® includes a recommendation module named RecommendLinks which can render such a list. The RecommendLinks helper takes two parameters: the name of an ini file, and the name of a section within that file; it uses these to locate the list of recommendations to display. If no extra details are specified, it will look in the [RecommendLinks] section of searches.ini. Allowing configuration of the location of links means that the same RecommendLinks module can be used in different contexts to display different lists of links. To implement the example of a link to a “search tips” guide when a user performs a search with no results, we could simply add *default_noresults_recommend[] = RecommendLinks* to the [General] section of $VUFIND_LOCAL_DIR/config/vufind/searches.ini, and then, in the same file, add this to the [RecommendLinks] section: @@ -113,11 +113,11 @@ As in the previous example, the presentation of the links can be customized by o Additional Resources --------------------- -VuFind’s recommendation module wiki page can be found at https://vufind.org/wiki/development:plugins:recommendation_modules. +VuFind®’s recommendation module wiki page can be found at https://vufind.org/wiki/development:plugins:recommendation_modules. Summary ------- -Recommendation modules are used by VuFind to display supplemental information that complements search results. They are highly configurable, so you can use them to communicate important information specific to certain search backends and/or search handlers. +Recommendation modules are used by VuFind® to display supplemental information that complements search results. They are highly configurable, so you can use them to communicate important information specific to certain search backends and/or search handlers. Review Questions ---------------- diff --git a/source/15-other_search_backends.rst b/source/15-other_search_backends.rst index b0b2a6b..93aab30 100644 --- a/source/15-other_search_backends.rst +++ b/source/15-other_search_backends.rst @@ -7,71 +7,71 @@ Learning Objectives After reading this chapter, you will understand: -• How to integrate VuFind with non-Solr-based search backends. -• Which commercial services might make useful complements to VuFind. -• The difference between VuFind and a “web-scale discovery service.” +• How to integrate VuFind® with non-Solr-based search backends. +• Which commercial services might make useful complements to VuFind®. +• The difference between VuFind® and a “web-scale discovery service.” 15.1 Introduction to Search Backends ------------------------------------ -As was previously discussed in section 4.5.2, VuFind refers to the bundles of code it uses to communicate with various search systems as “search backends.” VuFind’s backends are designed to be as interoperable as possible, so they can share common templates and tools (like recommendation modules); this design provides for a more consistent user experience and also reduces the amount of programming work needed to provide flexible, customizable user interfaces for a variety of systems. +As was previously discussed in section 4.5.2, VuFind® refers to the bundles of code it uses to communicate with various search systems as “search backends.” VuFind®’s backends are designed to be as interoperable as possible, so they can share common templates and tools (like recommendation modules); this design provides for a more consistent user experience and also reduces the amount of programming work needed to provide flexible, customizable user interfaces for a variety of systems. -So far in this book, the examples have focused on backends that use VuFind’s local Solr index. However, VuFind also includes other backends for communicating with a variety of systems (mostly subscription-based) which can be used to provide even more options to your users. This chapter describes those alternatives in more detail. +So far in this book, the examples have focused on backends that use VuFind®’s local Solr index. However, VuFind® also includes other backends for communicating with a variety of systems (mostly subscription-based) which can be used to provide even more options to your users. This chapter describes those alternatives in more detail. 15.2 Web-Scale Discovery Services ---------------------------------- -VuFind is often referred to as a “discovery system,” since it is software that helps users discover and explore resources. However, this sometimes causes confusion, because there are a variety of commercial indexes that also refer to themselves as “discovery services,” yet they are not exactly the same thing. VuFind is a user interface that can have data added to it; “web-scale discovery services” are subscription-based indexes that provide huge, pre-built aggregated indexes along with interfaces and APIs providing search access to those indexes. +VuFind® is often referred to as a “discovery system,” since it is software that helps users discover and explore resources. However, this sometimes causes confusion, because there are a variety of commercial indexes that also refer to themselves as “discovery services,” yet they are not exactly the same thing. VuFind® is a user interface that can have data added to it; “web-scale discovery services” are subscription-based indexes that provide huge, pre-built aggregated indexes along with interfaces and APIs providing search access to those indexes. -A commercial discovery system’s native interface may look a lot like VuFind, but it is not quite the same: because VuFind is an open source package that you can configure and customize on your own, it offers more flexibility and local control than a commercial discovery service. Conversely, the commercial discovery services often have access to data that individual users cannot easily obtain for indexing into their VuFind instances, and managing an index of that scale is a huge job that an individual library would be unlikely to wish to take on, so these services may have an advantage in terms of data availability. +A commercial discovery system’s native interface may look a lot like VuFind®, but it is not quite the same: because VuFind® is an open source package that you can configure and customize on your own, it offers more flexibility and local control than a commercial discovery service. Conversely, the commercial discovery services often have access to data that individual users cannot easily obtain for indexing into their VuFind® instances, and managing an index of that scale is a huge job that an individual library would be unlikely to wish to take on, so these services may have an advantage in terms of data availability. -Fortunately, it is possible to reach a “best of both worlds” compromise: if your library can afford to subscribe to a commercial discovery service, VuFind most likely has a search backend which can use that service’s API to pull search results directly into VuFind’s interface. This means that you can leverage the rich indexing work done by the commercial provider while also customizing the look and feel using VuFind’s built-in flexibility. +Fortunately, it is possible to reach a “best of both worlds” compromise: if your library can afford to subscribe to a commercial discovery service, VuFind® most likely has a search backend which can use that service’s API to pull search results directly into VuFind®’s interface. This means that you can leverage the rich indexing work done by the commercial provider while also customizing the look and feel using VuFind®’s built-in flexibility. -As of this writing, most of the major commercial discovery systems are supported by VuFind. EDS (EBSCO Discovery Service), Primo Central and Summon support are part of the core package; work in progress on WorldCat Discovery exists, and could be brought up to date and completed if sufficient interest were expressed. To use any of these services, it is simply a matter of adding some settings to the appropriate .ini file (and, in some cases, config.ini) as discussed in section 4.5.2. The service can then be combined with a local Solr index using the techniques described in chapter 13 and section 14.3. If you do not need to maintain a local index and instead wish to use one of these services as your primary search method, the “defaultModule” setting in config.ini can be used to switch VuFind’s default behavior – for example, you could change this from “Search” to “Summon” to make Summon the default home page of your VuFind instance. +As of this writing, most of the major commercial discovery systems are supported by VuFind®. EDS (EBSCO Discovery Service), Primo Central and Summon support are part of the core package; work in progress on WorldCat Discovery exists, and could be brought up to date and completed if sufficient interest were expressed. To use any of these services, it is simply a matter of adding some settings to the appropriate .ini file (and, in some cases, config.ini) as discussed in section 4.5.2. The service can then be combined with a local Solr index using the techniques described in chapter 13 and section 14.3. If you do not need to maintain a local index and instead wish to use one of these services as your primary search method, the “defaultModule” setting in config.ini can be used to switch VuFind®’s default behavior – for example, you could change this from “Search” to “Summon” to make Summon the default home page of your VuFind® instance. 15.3 Other Supported Services ----------------------------- -While web-scale discovery may offer the broadest range of additional search results in a VuFind instance, there are a number of additional supported services that may provide useful additions to your VuFind setup. +While web-scale discovery may offer the broadest range of additional search results in a VuFind® instance, there are a number of additional supported services that may provide useful additions to your VuFind® setup. 15.3.1 WorldCat _______________ -Many libraries of all sizes participate in OCLC’s services for copy cataloging and interlibrary loan. OCLC members have access to the WorldCat API, which makes it possible to search the organization’s union catalog and identify which libraries hold which books. VuFind includes a WorldCat backend which can be very useful for implementing a “books held by other libraries” area on a combined search screen. To set this up, you will need to get an API key from OCLC, and then you can fill in the [WorldCat] section of config.ini and the WorldCat.ini file to set things up. +Many libraries of all sizes participate in OCLC’s services for copy cataloging and interlibrary loan. OCLC members have access to the WorldCat API, which makes it possible to search the organization’s union catalog and identify which libraries hold which books. VuFind® includes a WorldCat backend which can be very useful for implementing a “books held by other libraries” area on a combined search screen. To set this up, you will need to get an API key from OCLC, and then you can fill in the [WorldCat] section of config.ini and the WorldCat.ini file to set things up. 15.3.2 BrowZine _______________ -BrowZine is a subscription service primarily designed to provide users with a convenient way to browse recent periodicals online. It also includes an API that can be useful for searching for specific journal titles held by the library. VuFind includes a backend that allows this journal title search to be incorporated into the system. +BrowZine is a subscription service primarily designed to provide users with a convenient way to browse recent periodicals online. It also includes an API that can be useful for searching for specific journal titles held by the library. VuFind® includes a backend that allows this journal title search to be incorporated into the system. 15.3.3 EIT (EBSCO Integration Toolkit) ______________________________________ -In addition to their web-scale discovery service, EDS, EBSCO also provides the less feature-rich “EBSCO Integration Toolkit,” which provides API-based search access to EBSCO-hosted databases. If your library has EBSCO subscriptions but has chosen not to subscribe to a full web-scale discovery service, EIT may provide a way to expose some database results through VuFind. Note that there has been discussion in the past about eventually replacing EIT with a different service, so this option may not be available indefinitely. +In addition to their web-scale discovery service, EDS, EBSCO also provides the less feature-rich “EBSCO Integration Toolkit,” which provides API-based search access to EBSCO-hosted databases. If your library has EBSCO subscriptions but has chosen not to subscribe to a full web-scale discovery service, EIT may provide a way to expose some database results through VuFind®. Note that there has been discussion in the past about eventually replacing EIT with a different service, so this option may not be available indefinitely. 15.3.4 LibGuides ________________ -The popular LibGuides service, used for developing library subject guides and other similar resources, provides an API for searching available guides. VuFind is able to use this as a search backend, making it possible to highlight relevant guides in a combined search. Note that it may also be possible to index LibGuides pages into a local web index using a sitemap as described in chapter 12; however, direct API access might be preferable if it is important to you to reflect page changes in real-time, if you do not wish to maintain a web index, or if you want to highlight guides independently of other web content. +The popular LibGuides service, used for developing library subject guides and other similar resources, provides an API for searching available guides. VuFind® is able to use this as a search backend, making it possible to highlight relevant guides in a combined search. Note that it may also be possible to index LibGuides pages into a local web index using a sitemap as described in chapter 12; however, direct API access might be preferable if it is important to you to reflect page changes in real-time, if you do not wish to maintain a web index, or if you want to highlight guides independently of other web content. 15.3.5 Pazpar2 ______________ -Pazpar2 is an open source federated search application which can be configured to perform a search across multiple databases and applications and then return a unified list of search results. VuFind can then be configured to fetch results from Pazpar2 through a search backend. Federated search is no longer a popular technology, due both to performance limitations and the complexity of configuring and maintaining it. Due to inherent limitations of Pazpar2’s features, the integration with VuFind lacks some functionality found in other backends (such as the ability to save records to favorites). For all of these reasons, using Pazpar2 is not recommended; however, support exists for those rare situations where it may be the best solution to a problem. +Pazpar2 is an open source federated search application which can be configured to perform a search across multiple databases and applications and then return a unified list of search results. VuFind® can then be configured to fetch results from Pazpar2 through a search backend. Federated search is no longer a popular technology, due both to performance limitations and the complexity of configuring and maintaining it. Due to inherent limitations of Pazpar2’s features, the integration with VuFind® lacks some functionality found in other backends (such as the ability to save records to favorites). For all of these reasons, using Pazpar2 is not recommended; however, support exists for those rare situations where it may be the best solution to a problem. Additional Resources --------------------- -The third-party content page in VuFind’s wiki should list all of the currently-supported backends: https://vufind.org/wiki/configuration:third-party_content. To learn more about the process of developing a whole new search backend, you can refer to this wiki page: https://vufind.org/wiki/development:howtos:connecting_a_new_external_data_source. +The third-party content page in VuFind®’s wiki should list all of the currently-supported backends: https://vufind.org/wiki/configuration:third-party_content. To learn more about the process of developing a whole new search backend, you can refer to this wiki page: https://vufind.org/wiki/development:howtos:connecting_a_new_external_data_source. Summary ------- -In addition to presenting content from a locally-maintained Solr index, VuFind can also provide search access to a variety of third-party systems, including “web-scale discovery.” By combining VuFind’s inherent flexibility with commercial services that provide data and functionality beyond the capabilities of your local team, you can develop a “best of both worlds” discovery experience for your users +In addition to presenting content from a locally-maintained Solr index, VuFind® can also provide search access to a variety of third-party systems, including “web-scale discovery.” By combining VuFind®’s inherent flexibility with commercial services that provide data and functionality beyond the capabilities of your local team, you can develop a “best of both worlds” discovery experience for your users Review Questions ---------------- -1. How do you change the default search presented by VuFind (e.g. replace Solr with Primo Central or EDS)? -2. Which web-scale discovery services are supported by VuFind, and how can they be configured? +1. How do you change the default search presented by VuFind® (e.g. replace Solr with Primo Central or EDS)? +2. Which web-scale discovery services are supported by VuFind®, and how can they be configured? 3. Which search backend is most useful for listing books held by other libraries? diff --git a/source/16-introduction_to_laminas.rst b/source/16-introduction_to_laminas.rst index 99770a0..8c92995 100644 --- a/source/16-introduction_to_laminas.rst +++ b/source/16-introduction_to_laminas.rst @@ -4,8 +4,8 @@ Chapter 16. Introduction to Laminas After reading this chapter, you will understand: -• Why VuFind uses a framework. -• How VuFind’s architecture is designed. +• Why VuFind® uses a framework. +• How VuFind®’s architecture is designed. • The importance of dependency injection. @@ -14,24 +14,24 @@ After reading this chapter, you will understand: To a new programmer, the biggest challenge of software development seems to be developing logic to solve problems. However, over time, an even bigger challenge is revealed: figuring out how to name and organize code so that it is easy to understand and maintain. One common way to reduce the burden of code organization is to pick a pre-existing framework that provides conventions and best practices for application design. Using a framework helps developers decide how to break down problems and arrange files, and it provides a common language across projects that share the same framework, making it easier to bring new developers on board. -When VuFind was first developed, it had a simple framework of sorts, but it was architecture built specifically for the development of VuFind. Over time, this original code became difficult to maintain; a lot of shared logic was copied-and-pasted between files rather than abstracted in a more reusable way, and most of the application’s files were dumped into a small number of directories, making it hard to find things. When the time came to develop a 2.0 release, rather than try to “fix” the existing system, the VuFind community decided to try rebuilding the application on an established open source framework, both to take advantage of the aforementioned advantages of a framework and to bring the software more in line with best practices of the time. While this proved to be a complex and laborious undertaking, it paid off in the long run by improving code quality and making subsequent extensions and customizations much easier to develop. +When VuFind® was first developed, it had a simple framework of sorts, but it was architecture built specifically for the development of VuFind®. Over time, this original code became difficult to maintain; a lot of shared logic was copied-and-pasted between files rather than abstracted in a more reusable way, and most of the application’s files were dumped into a small number of directories, making it hard to find things. When the time came to develop a 2.0 release, rather than try to “fix” the existing system, the VuFind® community decided to try rebuilding the application on an established open source framework, both to take advantage of the aforementioned advantages of a framework and to bring the software more in line with best practices of the time. While this proved to be a complex and laborious undertaking, it paid off in the long run by improving code quality and making subsequent extensions and customizations much easier to develop. -Several possible frameworks were considered for VuFind, but ultimately Zend Framework 2 was chosen, both because it was cutting-edge at the time (taking full advantage of then-new PHP features like namespaces) and because it provided very strong support for overriding core functionality, theming, and adding plug-ins. Zend Framework 2 never became the most popular PHP framework broadly, in large part due to its high complexity compared to some of its competitors, but its complexity is exactly what provided the flexibility needed to make it a perfect fit for VuFind. +Several possible frameworks were considered for VuFind®, but ultimately Zend Framework 2 was chosen, both because it was cutting-edge at the time (taking full advantage of then-new PHP features like namespaces) and because it provided very strong support for overriding core functionality, theming, and adding plug-ins. Zend Framework 2 never became the most popular PHP framework broadly, in large part due to its high complexity compared to some of its competitors, but its complexity is exactly what provided the flexibility needed to make it a perfect fit for VuFind®. -Over the years since VuFind first rebuilt on Zend Framework 2, the PHP community has moved forward and things have changed. The once-monolithic Zend Framework 2 evolved into Zend Framework 3, which was split into smaller, more independently-reusable components as the Composer dependency management tool gained popularity and made it easier to mix and match third-party PHP code. Eventually, Zend Framework rebranded itself as Laminas. As of VuFind 7.0, all references to “Zend” in the VuFind code have been replaced with equivalent Laminas code, and some Zend components have been replaced with better tools from other developers. However, in spite of the many things that have changed over the years, the basic ideas behind Zend Framework 2 are still a very important part of VuFind and have supported its ongoing success. +Over the years since VuFind® first rebuilt on Zend Framework 2, the PHP community has moved forward and things have changed. The once-monolithic Zend Framework 2 evolved into Zend Framework 3, which was split into smaller, more independently-reusable components as the Composer dependency management tool gained popularity and made it easier to mix and match third-party PHP code. Eventually, Zend Framework rebranded itself as Laminas. As of VuFind® 7.0, all references to “Zend” in the VuFind® code have been replaced with equivalent Laminas code, and some Zend components have been replaced with better tools from other developers. However, in spite of the many things that have changed over the years, the basic ideas behind Zend Framework 2 are still a very important part of VuFind® and have supported its ongoing success. -While migrating VuFind to a framework was a significant challenge, once the migration was complete and the framework’s basic conventions were understood, customizing and extending the software became quite simple. The VuFind team has done the hard work, so now end users can reap the benefits. This chapter provides background on the key concepts needed to understand and take advantage of VuFind’s design. It assumes a basic understanding of object-oriented programming with PHP; exploring those fundamentals is beyond the scope of this book, but many other books and online tutorials exist to fill that gap (see Additional Resources below for some suggestions). +While migrating VuFind® to a framework was a significant challenge, once the migration was complete and the framework’s basic conventions were understood, customizing and extending the software became quite simple. The VuFind® team has done the hard work, so now end users can reap the benefits. This chapter provides background on the key concepts needed to understand and take advantage of VuFind®’s design. It assumes a basic understanding of object-oriented programming with PHP; exploring those fundamentals is beyond the scope of this book, but many other books and online tutorials exist to fill that gap (see Additional Resources below for some suggestions). 16.2 Model-View-Controller Architecture (laminas-mvc) ----------------------------------------------------- -VuFind uses a model-view-controller architecture. The idea of this architecture is to separate code into three distinct parts: models, which represent data; views, which are used to display information to the end user; and controllers, which implement the “business logic” that makes the application actually do things. +VuFind® uses a model-view-controller architecture. The idea of this architecture is to separate code into three distinct parts: models, which represent data; views, which are used to display information to the end user; and controllers, which implement the “business logic” that makes the application actually do things. -This is a useful separation, because it allows key aspects of the application to be developed, tested and changed independently of one another. For example, best practices for web design change at a fairly rapid pace; by separating view logic from other parts of the application, it is possible to periodically refresh and modernize VuFind’s interface by changing all of its views, without having to worry about any of the code in the models or controllers. Similarly, the model/controller distinction separates code that deals with HOW to do things, from code about WHAT to do, which tends to improve the clarity of the code. +This is a useful separation, because it allows key aspects of the application to be developed, tested and changed independently of one another. For example, best practices for web design change at a fairly rapid pace; by separating view logic from other parts of the application, it is possible to periodically refresh and modernize VuFind®’s interface by changing all of its views, without having to worry about any of the code in the models or controllers. Similarly, the model/controller distinction separates code that deals with HOW to do things, from code about WHAT to do, which tends to improve the clarity of the code. -Of course, “model-view-controller” is a theoretical ideal, and VuFind is a real-world application, so every piece of code in VuFind does not necessarily fit neatly into a “model,” “view,” or “controller” bucket. In particular, the line between models and controllers can blur fairly easily. However, the most important idea here is “separation of concerns,” and VuFind’s design aims to split code into logical (and, in most cases, overridable) chunks that work together well but provide clear boundaries between one another. +Of course, “model-view-controller” is a theoretical ideal, and VuFind® is a real-world application, so every piece of code in VuFind® does not necessarily fit neatly into a “model,” “view,” or “controller” bucket. In particular, the line between models and controllers can blur fairly easily. However, the most important idea here is “separation of concerns,” and VuFind®’s design aims to split code into logical (and, in most cases, overridable) chunks that work together well but provide clear boundaries between one another. -VuFind’s specific implementation of model-view-controller uses the Laminas implementation of the architecture (laminas-mvc), with a few VuFind-specific customizations to add better support for themes. The views are all located in the $VUFIND_HOME/themes directory, as discussed in chapter 7. The models and controllers are all defined in PHP classes within VuFind’s code, and all are set up using configuration files in VuFind’s modules, leveraging the Laminas service manager. Both modules and the service manager are discussed in more detail below. Controllers will be given more attention in chapter 18. +VuFind®’s specific implementation of model-view-controller uses the Laminas implementation of the architecture (laminas-mvc), with a few VuFind®-specific customizations to add better support for themes. The views are all located in the $VUFIND_HOME/themes directory, as discussed in chapter 7. The models and controllers are all defined in PHP classes within VuFind®’s code, and all are set up using configuration files in VuFind®’s modules, leveraging the Laminas service manager. Both modules and the service manager are discussed in more detail below. Controllers will be given more attention in chapter 18. 16.3 Core Modules (laminas-modulemanager) ----------------------------------------- @@ -43,26 +43,26 @@ _______________________ In general terms, it is almost always beneficial to organize code into logical units, where closely related code is grouped together and unrelated code is separated. Laminas modules provide one mechanism for this type of organization. -From VuFind’s perspective, though, the most exciting feature of Laminas modules is the way they allow merging of configuration. When you load modules into an application in a particular order, the configuration settings from later modules are merged with, or fully override, settings from earlier modules. This is ideal for an application like VuFind, where there is a core set of defaults, but users may wish to program their own changes. In VuFind, users define all of their local code and application-level configuration within their own module, and this module is loaded last so that it can override VuFind’s defaults. This encourages developers to program customizations in a way that clearly expresses how their functionality differs from the core, and to change only the pieces that really need changing. This approach can make long-term maintenance easier, since a new developer taking over a project can likely make more sense of a discrete custom code module than they could a collection of minor edits hidden within the core code of the project. +From VuFind®’s perspective, though, the most exciting feature of Laminas modules is the way they allow merging of configuration. When you load modules into an application in a particular order, the configuration settings from later modules are merged with, or fully override, settings from earlier modules. This is ideal for an application like VuFind®, where there is a core set of defaults, but users may wish to program their own changes. In VuFind®, users define all of their local code and application-level configuration within their own module, and this module is loaded last so that it can override VuFind®’s defaults. This encourages developers to program customizations in a way that clearly expresses how their functionality differs from the core, and to change only the pieces that really need changing. This approach can make long-term maintenance easier, since a new developer taking over a project can likely make more sense of a discrete custom code module than they could a collection of minor edits hidden within the core code of the project. -Code modules are also interesting because they can be turned on or off conditionally within the code of the application. VuFind takes advantage of this in a couple of ways. First of all, some modules, like the administration panel, are only loaded when certain configuration settings are turned on. This reduces the odds of sensitive functionality being accessed by accident – if the code is completely inaccessible to the application, there are fewer potential attack vectors against it. Additionally, since modules can be loaded based on configuration, this provides part of the foundation of VuFind’s “multi-tenant” functionality, where a single installation of the software can serve up differently-configured instances based on the incoming URL. +Code modules are also interesting because they can be turned on or off conditionally within the code of the application. VuFind® takes advantage of this in a couple of ways. First of all, some modules, like the administration panel, are only loaded when certain configuration settings are turned on. This reduces the odds of sensitive functionality being accessed by accident – if the code is completely inaccessible to the application, there are fewer potential attack vectors against it. Additionally, since modules can be loaded based on configuration, this provides part of the foundation of VuFind®’s “multi-tenant” functionality, where a single installation of the software can serve up differently-configured instances based on the incoming URL. -Code modules are also a way of sharing functionality between Laminas-based applications; for example, VuFind makes use of a third-party module for integrating the Whoops error handler, a useful tool for debugging run-time errors. +Code modules are also a way of sharing functionality between Laminas-based applications; for example, VuFind® makes use of a third-party module for integrating the Whoops error handler, a useful tool for debugging run-time errors. -If you are interested in seeing exactly how VuFind loads its modules, take a look at the $VUFIND_HOME/config/application.config.php file, which contains most of the high-level Laminas application setup logic. +If you are interested in seeing exactly how VuFind® loads its modules, take a look at the $VUFIND_HOME/config/application.config.php file, which contains most of the high-level Laminas application setup logic. -16.3.2 VuFind’s Built-In Modules +16.3.2 VuFind®’s Built-In Modules ________________________________ -VuFind contains several code modules, found under the $VUFIND_HOME/modules directory; as of release 7.0, they are: +VuFind® contains several code modules, found under the $VUFIND_HOME/modules directory; as of release 7.0, they are: -• VuFind – the core of VuFind, containing the majority of the code. -• VuFindAdmin – code for VuFind’s web-based administration panel. -• VuFindApi – code for exposing VuFind search results and records through an API. -• VuFindConsole – code for all of VuFind’s command-line utilities. -• VuFindDevTools – code for utilities used by VuFind developers (only active when development mode is turned on in $VUFIND_LOCAL_DIR/httpd-vufind.conf). -• VuFindSearch – code for VuFind’s search system (including low-level backend code). -• VuFindTheme – code for VuFind’s theme system. +• VuFind – the core of VuFind®, containing the majority of the code. +• VuFindAdmin – code for VuFind®’s web-based administration panel. +• VuFindApi – code for exposing VuFind® search results and records through an API. +• VuFindConsole – code for all of VuFind®’s command-line utilities. +• VuFindDevTools – code for utilities used by VuFind® developers (only active when development mode is turned on in $VUFIND_LOCAL_DIR/httpd-vufind.conf). +• VuFindSearch – code for VuFind®’s search system (including low-level backend code). +• VuFindTheme – code for VuFind®’s theme system. There is also a module called VuFindLocalTemplate, which is not used directly but is the basis for building local modules, as described in the next section. @@ -70,20 +70,20 @@ There is also a module called VuFindLocalTemplate, which is not used directly bu 16.3.3 Local Custom Modules ___________________________ -Most VuFind installations have at most one local code module; many have none at all. If you install VuFind using the Debian package as described in section 2.2, no module will be set up by default. Fortunately, adding a module is fairly straightforward and can be done either manually or automatically. +Most VuFind® installations have at most one local code module; many have none at all. If you install VuFind® using the Debian package as described in section 2.2, no module will be set up by default. Fortunately, adding a module is fairly straightforward and can be done either manually or automatically. To do it manually, just copy the $VUFIND_HOME/VuFindLocalTemplate directory and its contents to a new directory name. Edit the namespace declarations in both PHP files (Module.php and config/module.config.php) within your new module to match the name of the module’s directory. Then, edit $VUFIND_LOCAL_DIR/httpd-vufind.conf, uncomment the VUFIND_LOCAL_MODULES environment variable line, and insert the appropriate module name. Finally, restart Apache so the change takes effect. -To do it automatically, run php $VUFIND_HOME/install.php – the VuFind install script asks a question about local modules and automatically copies and configures VuFindLocalTemplate for you based on your answer. Just be sure you answer the other questions in the script consistently with your current configuration so that you don’t break anything. If something goes wrong, don’t worry – the install script backs up your $VUFIND_LOCAL_DIR/httpd-vufind.conf to a filename with a timestamped extension, so you can copy it back to fix any problems. +To do it automatically, run php $VUFIND_HOME/install.php – the VuFind® install script asks a question about local modules and automatically copies and configures VuFindLocalTemplate for you based on your answer. Just be sure you answer the other questions in the script consistently with your current configuration so that you don’t break anything. If something goes wrong, don’t worry – the install script backs up your $VUFIND_LOCAL_DIR/httpd-vufind.conf to a filename with a timestamped extension, so you can copy it back to fix any problems. -Also note that, once you are using a local module, you will also want to export its name into the $VUFIND_LOCAL_MODULES environment variable whenever you are working with VuFind’s command- line tools to be sure your custom code is properly accounted for. If you installed VuFind using standard practices, you can most easily ensure this variable is set correctly by adding a line to the /etc/profile.d/vufind.sh file where $VUFIND_HOME and $VUFIND_LOCAL_DIR should already be getting set up. Exporting the local module name is especially important when you are working with code generators, since they need to be able to find your module in order to add code to it; see section 9.3 for an example of code generators in action. +Also note that, once you are using a local module, you will also want to export its name into the $VUFIND_LOCAL_MODULES environment variable whenever you are working with VuFind®’s command- line tools to be sure your custom code is properly accounted for. If you installed VuFind® using standard practices, you can most easily ensure this variable is set correctly by adding a line to the /etc/profile.d/vufind.sh file where $VUFIND_HOME and $VUFIND_LOCAL_DIR should already be getting set up. Exporting the local module name is especially important when you are working with code generators, since they need to be able to find your module in order to add code to it; see section 9.3 for an example of code generators in action. 16.4 Dependency Injection and Containers (laminas-servicemanager) ----------------------------------------------------------------- -As noted earlier, deciding how to break an application into parts, and where to put those parts, is a big challenge. Another related challenge is figuring out how to let those parts interact with one another. It is very common for software components to have dependencies on one another, but if these relationships are not managed carefully, code can become hard to maintain and customize. To address this particular problem, VuFind follows Laminas’ lead and uses dependency injection. +As noted earlier, deciding how to break an application into parts, and where to put those parts, is a big challenge. Another related challenge is figuring out how to let those parts interact with one another. It is very common for software components to have dependencies on one another, but if these relationships are not managed carefully, code can become hard to maintain and customize. To address this particular problem, VuFind® follows Laminas’ lead and uses dependency injection. -The idea of dependency injection is simple: every software component should declare up front what other components it depends upon for its functionality. When the component is constructed, those dependencies should be injected into it. For example, VuFind contains code that talks to Solr over HTTP. In order to do this, it needs access to code that speaks the HTTP protocol. When we build the Solr connector, we inject the HTTP client so it has access to the functionality that it needs. +The idea of dependency injection is simple: every software component should declare up front what other components it depends upon for its functionality. When the component is constructed, those dependencies should be injected into it. For example, VuFind® contains code that talks to Solr over HTTP. In order to do this, it needs access to code that speaks the HTTP protocol. When we build the Solr connector, we inject the HTTP client so it has access to the functionality that it needs. This leads to a sort of “inside out” feeling to the code – you have to build the more specific, low-level components first in order to pass them along to the higher-level components that depend upon them. But consider the alternative: we could have each component essentially build its own dependencies as it needs them. For example, rather than injecting an HTTP client into the Solr connector, we could have the Solr connector build its own HTTP client at the time that it needs to use it. However, the dependency injection approach offers some important advantages. @@ -95,7 +95,7 @@ The Laminas ServiceManager is a class whose responsibility is building and stori To make the most of the service manager, the application utilizing it needs to be programmed so that, instead of directly building objects, it always fetches them from the service manager. By introducing this extra layer of abstraction, we also gain a lot of power and flexibility. If we want to override a particular service, we can simply reconfigure the service manager to use our customized code instead of the normal service; we don’t have to change anything else, and we can be confident that our custom code will be used everywhere that the original code was used. To contrast, if an application used inline construction of objects instead of the service manager, replacing a frequently-used class might require changes to dozens or even hundreds of files and would be a source of maintenance headaches every time a new version of the software was released. -In addition to the power of abstraction, the service manager helps with dependency injection in a very important way. The service manager can build objects using a variety of strategies, but the most common (and the one used by most of VuFind’s code) is with the help of factory classes. As the name suggests, a factory is a class used for building other objects. The factories used by the Laminas service manager are powerful for two reasons: they can use the service manager itself as part of the process of building objects (greatly simplifying the job of managing nested dependencies), and they are reusable: as long as you adhere to certain conventions, like ensuring that service names in the service manager always correspond to PHP class names, the same factory can be used to build objects belonging to multiple classes. Reusability can be a real time saver when many objects in your application have the exact same set of dependencies. +In addition to the power of abstraction, the service manager helps with dependency injection in a very important way. The service manager can build objects using a variety of strategies, but the most common (and the one used by most of VuFind®’s code) is with the help of factory classes. As the name suggests, a factory is a class used for building other objects. The factories used by the Laminas service manager are powerful for two reasons: they can use the service manager itself as part of the process of building objects (greatly simplifying the job of managing nested dependencies), and they are reusable: as long as you adhere to certain conventions, like ensuring that service names in the service manager always correspond to PHP class names, the same factory can be used to build objects belonging to multiple classes. Reusability can be a real time saver when many objects in your application have the exact same set of dependencies. All of this should become clearer with an example – let’s implement the “Solr connector with an HTTP client dependency” example with Laminas\ServiceManager. We’ll assume for the sake of example that the HTTP client has no other dependencies, but the Solr connector needs the HTTP client. To do this, we would configure the ServiceManager with two services. First, the HTTP client could be configured to use the ServiceManager’s generic “InvokableFactory” – a reusable factory which builds an object in the simplest possible way, without passing any dependencies into it. Next, the Solr connector could be configured to use a new custom factory. In the Solr connector’s factory code, we could fetch the HTTP client from the service manager and then use it to build the Solr connector. Let’s look at how this would break down step by step: @@ -107,7 +107,7 @@ All of this should become clearer with an example – let’s implement the “S For a very complex object, the service manager might end up triggering a complex chain reaction, using many factories and looking up many additional services... but as long as each factory simply asks the service manager for the immediate dependencies of the object being built, the developer doesn’t need to worry too much about those implementation details. The chain may be complex, but each link in the chain is easy to build and clear to read. -Understanding the role of the service manager is probably the single most important key to understanding VuFind’s code. It is not necessarily obvious at first, but the power of this one tool informs much of VuFind’s design, both in how the code is broken down into parts and in how nearly everything can be customized and extended. If this is not yet clear, you may wish to revisit this chapter after you have had more experience working with the code. Chapter 17, discussing plug-ins, may shed additional light as well. +Understanding the role of the service manager is probably the single most important key to understanding VuFind®’s code. It is not necessarily obvious at first, but the power of this one tool informs much of VuFind®’s design, both in how the code is broken down into parts and in how nearly everything can be customized and extended. If this is not yet clear, you may wish to revisit this chapter after you have had more experience working with the code. Chapter 17, discussing plug-ins, may shed additional light as well. Additional Resources --------------------- @@ -119,16 +119,16 @@ The Wikipedia page on Model-View-Controller architecture provides some broader c You can learn more about the ecosystem of Laminas components at the project’s official website: https://getlaminas.org/. The Wikipedia page on Dependency Injection goes into greater depth on the concept: https://en.wikipedia.org/wiki/Dependency_injection. -VuFind’s Developer Manual contains more specific notes on architecture and best practices: https://vufind.org/wiki/development. +VuFind®’s Developer Manual contains more specific notes on architecture and best practices: https://vufind.org/wiki/development. The tutorial video at https://vufind.org/wiki/videos:code_generators_1 includes instructions on setting up your own local code module. Summary ------- -As an open source project designed to serve many users and many use cases, it is important for VuFind to follow shared best practices and to structure its code in a way that is easily understood and modified. The project uses Laminas components as its basis in order to meet these goals. Specifically, it uses the Laminas model-view-controller architecture to separate data, business logic and presentation templates, it uses the module manager to group related code together and to support configuration overrides, and it uses the service manager to allow easy class overriding and factory-based dependency injection. All of these tools work together to inform the shape of new code as the project grows and to provide easy hooks for developers to use while customizing local instances. +As an open source project designed to serve many users and many use cases, it is important for VuFind® to follow shared best practices and to structure its code in a way that is easily understood and modified. The project uses Laminas components as its basis in order to meet these goals. Specifically, it uses the Laminas model-view-controller architecture to separate data, business logic and presentation templates, it uses the module manager to group related code together and to support configuration overrides, and it uses the service manager to allow easy class overriding and factory-based dependency injection. All of these tools work together to inform the shape of new code as the project grows and to provide easy hooks for developers to use while customizing local instances. Review Questions ---------------- 1. What is the difference between a model, a view, and a controller? -2. What is one disadvantage of dependency injection, and how does VuFind mitigate it? -3. Name three Laminas components used by VuFind, and explain their significance. +2. What is one disadvantage of dependency injection, and how does VuFind® mitigate it? +3. Name three Laminas components used by VuFind®, and explain their significance. diff --git a/source/17-vufind_plug-ins.rst b/source/17-vufind_plug-ins.rst index 14c277c..3c0a954 100644 --- a/source/17-vufind_plug-ins.rst +++ b/source/17-vufind_plug-ins.rst @@ -1,22 +1,22 @@ -########################### -Chapter 17. VuFind Plug-Ins -########################### +############################ +Chapter 17. VuFind® Plug-Ins +############################ After reading this chapter, you will understand: -• How and why VuFind uses plug-in code. -• Which types of plug-ins are available in VuFind. +• How and why VuFind® uses plug-in code. +• Which types of plug-ins are available in VuFind®. • How to generate plug-ins on the command line. -17.1 VuFind’s Plug-In Architecture ----------------------------------- +17.1 VuFind®’s Plug-In Architecture +----------------------------------- -One of VuFind’s design goals is to be as extensible as possible, and one of the ways in which it implements extensibility is to use plug-in code. All of VuFind’s pluggable code follows the same basic strategy: define a common interface that broadly defines a piece of required functionality, offer multiple implementations of the interface to meet different needs, and use configuration to load the appropriate implementation based on user preferences. Plug-ins fall into two broad categories: those that allow a function to be fulfilled by different technologies by exposing a generic interface to higher-level VuFind code (for example, authentication handlers, session handlers, ILS drivers, etc.); and those that provide “hooks” for custom functionality (for example, recommendation modules, related record plug-ins, DOI handlers, etc). Regardless of the behavior or purpose of the plug-in, the general mechanisms for configuring and accessing it are the same. +One of VuFind®’s design goals is to be as extensible as possible, and one of the ways in which it implements extensibility is to use plug-in code. All of VuFind®’s pluggable code follows the same basic strategy: define a common interface that broadly defines a piece of required functionality, offer multiple implementations of the interface to meet different needs, and use configuration to load the appropriate implementation based on user preferences. Plug-ins fall into two broad categories: those that allow a function to be fulfilled by different technologies by exposing a generic interface to higher-level VuFind® code (for example, authentication handlers, session handlers, ILS drivers, etc.); and those that provide “hooks” for custom functionality (for example, recommendation modules, related record plug-ins, DOI handlers, etc). Regardless of the behavior or purpose of the plug-in, the general mechanisms for configuring and accessing it are the same. -As discussed in section 16.4, Laminas offers a service manager component which offers a useful abstraction for loading and building complex objects “on the fly” based on configuration. The service manager provides the foundation for all of VuFind’s plug-ins. +As discussed in section 16.4, Laminas offers a service manager component which offers a useful abstraction for loading and building complex objects “on the fly” based on configuration. The service manager provides the foundation for all of VuFind®’s plug-ins. -VuFind actually uses a large number of service managers, arranged in a two-level hierarchy. There is the top-level service manager which manages unique and globally-used services as well as all of the lower- level service managers, each of which in turn manages a family of related plug-ins. To fetch a specific plug-in, code first needs to request the appropriate plug-in manager from the top-level service manager, and then the plug-in itself can be requested from the plug-in manager. To differentiate between the two different levels of managers, the lower-level managers are usually referred to as “plug-in managers” while the top-level manager is simply called “the service manager.” +VuFind® actually uses a large number of service managers, arranged in a two-level hierarchy. There is the top-level service manager which manages unique and globally-used services as well as all of the lower- level service managers, each of which in turn manages a family of related plug-ins. To fetch a specific plug-in, code first needs to request the appropriate plug-in manager from the top-level service manager, and then the plug-in itself can be requested from the plug-in manager. To differentiate between the two different levels of managers, the lower-level managers are usually referred to as “plug-in managers” while the top-level manager is simply called “the service manager.” The most meaningful difference between the top-level service manager and a lower-level plug-in manager is simply how each is configured; they all provide the same dependency injection and container functionality, but they are set up to build and manage different types of objects. Using a dedicated plug- in manager for each type of plug-in offers several benefits over simply loading everything into the top- level service manager: @@ -27,16 +27,16 @@ The most meaningful difference between the top-level service manager and a lower • For services that need to dynamically load a particular type of plug-in, we can inject a specific plug-in manager as a dependency rather than injecting the top-level service manager, which is a bad design practice (because it offers more access to resources than any single class is likely to need, making relationships between parts of the application less clear). -All of the plug-in managers are registered as services in the top-level service manager, using their class names as their service names. All of the plug-ins are registered in the corresponding plug-in managers using both their class names and shorter, more human-readable aliases (which make them easier to load based on configuration). Each plug-in manager is implemented as a subclass of the Laminas service manager, with default configurations built into the code. These default configurations can be overridden and extended through local module configuration, making it possible to easily add or replace any kind of plug-in. The command line generator tools (discussed in the next section) can be used to automate most of the configuration work to take advantage of this setup. A few plug-in types are actually part of the Laminas framework (controllers, controller plug-ins and view helpers); the rest are specific to VuFind. +All of the plug-in managers are registered as services in the top-level service manager, using their class names as their service names. All of the plug-ins are registered in the corresponding plug-in managers using both their class names and shorter, more human-readable aliases (which make them easier to load based on configuration). Each plug-in manager is implemented as a subclass of the Laminas service manager, with default configurations built into the code. These default configurations can be overridden and extended through local module configuration, making it possible to easily add or replace any kind of plug-in. The command line generator tools (discussed in the next section) can be used to automate most of the configuration work to take advantage of this setup. A few plug-in types are actually part of the Laminas framework (controllers, controller plug-ins and view helpers); the rest are specific to VuFind®. -Several components of VuFind discussed elsewhere in this text utilize this plug-in architecture: view helpers (section 7.5), record drivers (section 9.1), recommendation modules (chapter 14), and most of the components that make up search backends (chapter 15). There are many more plug-in types within the VuFind code; see “Additional Resources” below to learn more about them. +Several components of VuFind® discussed elsewhere in this text utilize this plug-in architecture: view helpers (section 7.5), record drivers (section 9.1), recommendation modules (chapter 14), and most of the components that make up search backends (chapter 15). There are many more plug-in types within the VuFind® code; see “Additional Resources” below to learn more about them. -There is no question that VuFind’s plug-in manager and service manager code and configuration is complex, and it can take some time and experience to fully understand it. However, this complexity is designed to make a lot of higher-level tasks easier and simpler. Only experienced VuFind developers need to master this; newcomers can simply take advantage of it to build custom code more easily. Do not be concerned if you do not understand everything yet; you can take advantage of the power of plug- ins without fully understanding how they are implemented in VuFind’s core code. The next section will show you how to easily build your own plug-ins without having to touch any low-level VuFind code or configuration. +There is no question that VuFind®’s plug-in manager and service manager code and configuration is complex, and it can take some time and experience to fully understand it. However, this complexity is designed to make a lot of higher-level tasks easier and simpler. Only experienced VuFind® developers need to master this; newcomers can simply take advantage of it to build custom code more easily. Do not be concerned if you do not understand everything yet; you can take advantage of the power of plug- ins without fully understanding how they are implemented in VuFind®’s core code. The next section will show you how to easily build your own plug-ins without having to touch any low-level VuFind® code or configuration. 17.2 Command Line Plug-In Generator Tools ----------------------------------------- -While the Laminas service manager and plug-in managers are powerful tools, setting up or extending a plug-in can require a lot of work: creating the plug-in class itself, creating a factory to build the plug-in, and configuring everything in the module system. Fortunately, VuFind includes command-line tools that automate most of this work, ensuring that everything ends up in the right place, and allowing you to focus on the more interesting work of writing meaningful code. +While the Laminas service manager and plug-in managers are powerful tools, setting up or extending a plug-in can require a lot of work: creating the plug-in class itself, creating a factory to build the plug-in, and configuring everything in the module system. Fortunately, VuFind® includes command-line tools that automate most of this work, ensuring that everything ends up in the right place, and allowing you to focus on the more interesting work of writing meaningful code. There are a couple of things you’ll need to remember to make the most of the generators: @@ -49,7 +49,7 @@ There are a couple of things you’ll need to remember to make the most of the g 17.2.1 Replacing/Extending a Plugin (the “extendclass” generator) _________________________________________________________________ -If you want to extend or override an existing plug-in, VuFind’s “extendclass” generator will help. You simply provide it with the name of the core VuFind class that you wish to extend, and the name of a module where you would like to put the new subclass, and it will set up the rest. By default, it will reuse whatever factory is already defined in the core to build your new subclass, but if you also wish to create a new factory, you can add the “--extendfactory" option to create a subclass of the factory as well as the plug-in itself. +If you want to extend or override an existing plug-in, VuFind®’s “extendclass” generator will help. You simply provide it with the name of the core VuFind® class that you wish to extend, and the name of a module where you would like to put the new subclass, and it will set up the rest. By default, it will reuse whatever factory is already defined in the core to build your new subclass, but if you also wish to create a new factory, you can add the “--extendfactory" option to create a subclass of the factory as well as the plug-in itself. The examples below assume a local custom module with the name “MyModule.” @@ -71,7 +71,7 @@ See section 9.3.2 for another example of this code generator in action. 17.2.2 Creating a Plugin (the “plugin” generator) _________________________________________________ -If you want to create a new plug-in, VuFind’s “plugin” generator will do the job. You simply tell it the class name that you wish to create, and it will infer from the namespace of the class which module you want to update and which plug-in manager needs to be updated to register it. If you provide a class name by itself, the generator will also build an accompanying factory class to build the plug-in. If you provide the name of an existing factory as the command’s second parameter, that factory will be used to construct the object in the generated configuration, and no additional factory class will be built. +If you want to create a new plug-in, VuFind®’s “plugin” generator will do the job. You simply tell it the class name that you wish to create, and it will infer from the namespace of the class which module you want to update and which plug-in manager needs to be updated to register it. If you provide a class name by itself, the generator will also build an accompanying factory class to build the plug-in. If you provide the name of an existing factory as the command’s second parameter, that factory will be used to construct the object in the generated configuration, and no additional factory class will be built. The examples below assume a local custom module with the name “MyModule.” @@ -92,7 +92,7 @@ As discussed in chapter 14, recommendation modules provide a way to supplement s 17.3.1 Building the Recommendation Module Class _______________________________________________ -This example will assume that your local module is set up and named MyModule, and that the recommendation module class you want to create will be named MyModule\Recommend\LocalText. Every part of this name is meaningful to VuFind’s generator tool: the first part of the namespace (MyModule) tells it that the class needs to be created inside the MyModule module; the middle part of the namespace (Recommend) tells it to create a recommendation module, since every VuFind recommendation module is in the “VuFind\Recommend” namespace; the final part specifies the actual class name being created. +This example will assume that your local module is set up and named MyModule, and that the recommendation module class you want to create will be named MyModule\Recommend\LocalText. Every part of this name is meaningful to VuFind®’s generator tool: the first part of the namespace (MyModule) tells it that the class needs to be created inside the MyModule module; the middle part of the namespace (Recommend) tells it to create a recommendation module, since every VuFind® recommendation module is in the “VuFind\Recommend” namespace; the final part specifies the actual class name being created. Because our example is going to be very simple and will have no external dependencies, we do not need to build a custom factory for it. Instead, we want to use the standard, framework-provided Laminas\ServiceManager\Factory\InvokableFactory which (as discussed in section 16.4) simply constructs objects without passing any parameters to them. @@ -225,29 +225,29 @@ Now if you refresh the search results, “a librarian” will be replaced with In the LocalText PHP class, we added a property called $name. This is initialized to “a librarian,” so it has a default value. -The setConfig() method is called when the recommendation module is initialized; it is passed any configuration settings that are attached to the recommendation module name with a colon. In other words, when we configure “LocalText: Your Name Here” in searches.ini, VuFind passes the text “Your Name Here” to the setConfig method. The logic here checks if the incoming text is non-empty, and overrides the default value of the $name property when appropriate. +The setConfig() method is called when the recommendation module is initialized; it is passed any configuration settings that are attached to the recommendation module name with a colon. In other words, when we configure “LocalText: Your Name Here” in searches.ini, VuFind® passes the text “Your Name Here” to the setConfig method. The logic here checks if the incoming text is non-empty, and overrides the default value of the $name property when appropriate. We also defined a public getName() method, which simply provides the current value of the $name property. In the view template, the recommendation module object is exposed as a value called $this->recommend. Thus, we can define any public methods we like in the PHP code and then access those methods from the template. Thus, *escapeHtml($this->recommend->getName())?>* in the template provides the connection where the value from our configuration file can be surfaced in the user interface. The *$this->escapeHtml()* call wrapped around *$this->recommend->getName()* simply ensures that the provided text is properly formatted as HTML, in case it contains special characters like <, > or &. -Hopefully you can see how this offers you a great deal of power to use PHP code to retrieve information from various sources and then expose it to the user through the template. If you examine the existing recommendation modules that ship with VuFind, you will see more advanced examples of how to leverage third-party APIs, make decisions based on the contents of search results, and even manipulate the parameters used to perform the search. +Hopefully you can see how this offers you a great deal of power to use PHP code to retrieve information from various sources and then expose it to the user through the template. If you examine the existing recommendation modules that ship with VuFind®, you will see more advanced examples of how to leverage third-party APIs, make decisions based on the contents of search results, and even manipulate the parameters used to perform the search. Additional Resources -------------------- -The VuFind wiki has a page summarizing all available plug-in types and providing advice on how to build them: https://vufind.org/wiki/development:plugins. +The VuFind® wiki has a page summarizing all available plug-in types and providing advice on how to build them: https://vufind.org/wiki/development:plugins. The tutorial video at https://vufind.org/wiki/videos:code_generators_1 includes an example of building a custom recommendation module. Summary ------- -The Laminas service manager component provides conventions that allow VuFind to support extensible and pluggable functionality throughout the application. Nearly any piece of VuFind can be replaced by overriding configurations, and additional functionality can be plugged in in a similar way. Code generator tools included with the application automate the process of creating files and editing configurations, allowing developers to focus on writing code rather than setting up boilerplate. +The Laminas service manager component provides conventions that allow VuFind® to support extensible and pluggable functionality throughout the application. Nearly any piece of VuFind® can be replaced by overriding configurations, and additional functionality can be plugged in in a similar way. Code generator tools included with the application automate the process of creating files and editing configurations, allowing developers to focus on writing code rather than setting up boilerplate. Review Questions ---------------- -1. What are some reasons that VuFind uses plug-in managers instead of simply registering all plug-in code directly within the top-level service manager? +1. What are some reasons that VuFind® uses plug-in managers instead of simply registering all plug-in code directly within the top-level service manager? 2. In what situation would you want to generate a new factory for a plug-in? When would it be better to use an existing factory? 3. What is wrong with the following command? diff --git a/source/18-controllers_and_actions.rst b/source/18-controllers_and_actions.rst index 4ba5fc2..e97dcf1 100644 --- a/source/18-controllers_and_actions.rst +++ b/source/18-controllers_and_actions.rst @@ -13,7 +13,7 @@ After reading this chapter, you will understand: 18.1 VuFind’s Controllers and Actions -As discussed in section 16.2, VuFind uses the laminas-mvc package to implement a model-view-controller architecture. Controllers are the part of the code that make things happen in VuFind. +As discussed in section 16.2, VuFind® uses the laminas-mvc package to implement a model-view-controller architecture. Controllers are the part of the code that make things happen in VuFind®. When a user accesses one of VuFind’s URLs, the laminas-router system is triggered. The router component compares the text of the request URL against a set of rules configured in VuFind’s Laminas modules in order to figure out which piece of controller code to dispatch. @@ -40,11 +40,11 @@ This particular route uses a ‘type’ of ‘Laminas\Router\Http\Segment’, wh To learn about the Laminas router component in more detail, see the documentation at https://docs.laminas.dev/laminas-router/routing/. -To continue the example of accessing the URL of http://localhost/vufind/Example/Page, the default route defined above would extract a ‘controller’ value of ‘Example’ and an ‘action’ value of ‘Page’. These route values have special meanings when used in the laminas-mvc framework that VuFind is built on: they control which PHP code gets executed. +To continue the example of accessing the URL of http://localhost/vufind/Example/Page, the default route defined above would extract a ‘controller’ value of ‘Example’ and an ‘action’ value of ‘Page’. These route values have special meanings when used in the laminas-mvc framework that VuFind® is built on: they control which PHP code gets executed. -First, the controller value is used to look up a controller class in the controller plugin manager (see chapter 17 for more on plugin managers). By convention, VuFind’s controllers are set up with short names as aliases to make routing configuration more readable – thus, if VuFind had a VuFind\Controller\ExampleController, it would also have an alias configuration allowing ‘Example’ to be used as a shorthand for retrieving the controller from the plugin manager. +First, the controller value is used to look up a controller class in the controller plugin manager (see chapter 17 for more on plugin managers). By convention, VuFind’s controllers are set up with short names as aliases to make routing configuration more readable – thus, if VuFind® had a VuFind\Controller\ExampleController, it would also have an alias configuration allowing ‘Example’ to be used as a shorthand for retrieving the controller from the plugin manager. -If no matching controller can be found, VuFind will throw a 404 not found error… but if a controller exists, the action parameter from the route will be used to call a matching method on the controller. Controller actions must be named as an action name, followed by the word Action – so in the current example, VuFind would attempt to call the pageAction() method on the ExampleController. +If no matching controller can be found, VuFind® will throw a 404 not found error… but if a controller exists, the action parameter from the route will be used to call a matching method on the controller. Controller actions must be named as an action name, followed by the word Action – so in the current example, VuFind® would attempt to call the pageAction() method on the ExampleController. Controller action methods most commonly return a Laminas\View\Model\ViewModel object, which is a container that is used to pass data along to a view template for rendering. Most controller actions have corresponding view templates in the theme (for example, the pageAction of the ExampleController would expect to render a template named example/page.phtml). Any data elements that are inserted into the ViewModel object will be available to the view template as local variables that can be used for display. @@ -52,19 +52,19 @@ In some special cases, controller actions may instead return a Laminas\Http\Resp Controllers are discussed in more detail in the Laminas documentation at https://docs.laminas.dev/laminas-mvc/controllers/. -Most of VuFind’s controllers extend the VuFind\Controller\AbstractBase class, which provides some helpful convenience methods for common VuFind-related activities. For example, there is a createViewModel() method which constructs a ViewModel object with some useful pre-populated values. In general, when adding new functionality to VuFind, it is useful to examine existing controllers and actions providing similar functionality, as there may be some patterns observed there that can be usefully emulated. +Most of VuFind’s controllers extend the VuFind\Controller\AbstractBase class, which provides some helpful convenience methods for common VuFind®-related activities. For example, there is a createViewModel() method which constructs a ViewModel object with some useful pre-populated values. In general, when adding new functionality to VuFind®, it is useful to examine existing controllers and actions providing similar functionality, as there may be some patterns observed there that can be usefully emulated. -18.2 Example: Adding a Page to VuFind +18.2 Example: Adding a Page to VuFind® -------------------------------------- -The http://localhost/vufind/Example/Page action from the previous section would lead to a 404 error in most VuFind instances, since that page is not defined as part of the core VuFind package. However, with the help of code generators, it would be easy to define such a page. Since controllers are just another plug-in (as described in chapter 17), some parts of the process should seem familiar by now. This section will show all of the steps involved. +The http://localhost/vufind/Example/Page action from the previous section would lead to a 404 error in most VuFind® instances, since that page is not defined as part of the core VuFind® package. However, with the help of code generators, it would be easy to define such a page. Since controllers are just another plug-in (as described in chapter 17), some parts of the process should seem familiar by now. This section will show all of the steps involved. -For the purposes of this example, we will assume that you have a local code module called MyModule (see section 16.3.3 for more on local code modules) and a local theme called localtheme (see section 7.2 for more on theme generation). This example also assumes you are using VuFind 7.0 or later; earlier releases of VuFind did not support automatic generation of controllers, though of course the same effect could be achieved by manually creating files and editing configurations. +For the purposes of this example, we will assume that you have a local code module called MyModule (see section 16.3.3 for more on local code modules) and a local theme called localtheme (see section 7.2 for more on theme generation). This example also assumes you are using VuFind® 7.0 or later; earlier releases of VuFind® did not support automatic generation of controllers, though of course the same effect could be achieved by manually creating files and editing configurations. 18.2.1 Creating the ExampleController ------------------------------------- -VuFind’s plugin generator can create controllers starting with VuFind 7.0, so this command will establish the ExampleController and configure it to be built using the standard VuFind\Controller\AbstractBaseFactory: +VuFind’s plugin generator can create controllers starting with VuFind® 7.0, so this command will establish the ExampleController and configure it to be built using the standard VuFind\Controller\AbstractBaseFactory: .. code-block:: console @@ -101,12 +101,12 @@ By convention, the template file displayed by the pageAction of the ExampleContr

Hello, world!

-At this point, if you add “Example/Page” to your VuFind home URL, you should see “Hello, world!” in your browser. This is because of the default route definition discussed in the previous section. However, there are some advantages to defining a specific route for this action, so there’s one more step to go… +At this point, if you add “Example/Page” to your VuFind® home URL, you should see “Hello, world!” in your browser. This is because of the default route definition discussed in the previous section. However, there are some advantages to defining a specific route for this action, so there’s one more step to go… 18.2.3 Defining a Route ________________________ -For simple routes like this one, VuFind has a “staticroute” generator you can call: +For simple routes like this one, VuFind® has a “staticroute” generator you can call: .. code-block:: console @@ -126,20 +126,20 @@ Additional Resources As noted earlier, the Laminas documentation provides more detail on controllers and routing; see https://docs.laminas.dev/laminas-mvc/ and https://docs.laminas.dev/laminas-router/ in particular. -The VuFind wiki page on controllers also contains some VuFind-specific details and up-to-date examples: https://vufind.org/wiki/development:plugins:controllers. +The VuFind® wiki page on controllers also contains some VuFind®-specific details and up-to-date examples: https://vufind.org/wiki/development:plugins:controllers. The tutorial video at https://vufind.org/wiki/videos:code_generators_2 includes an example of building a local custom controller. Summary ------- -VuFind leverages the laminas-mvc system for model-view-controller architecture. The laminas-router component helps turn user request URLs into calls to action methods on controller classes. These methods process user input and help turn it into user output, either by generating HTTP responses or by delegating work to the view template system. Controllers are plug-ins, just like many other parts of VuFind, so they can be easily created and extended. +VuFind® leverages the laminas-mvc system for model-view-controller architecture. The laminas-router component helps turn user request URLs into calls to action methods on controller classes. These methods process user input and help turn it into user output, either by generating HTTP responses or by delegating work to the view template system. Controllers are plug-ins, just like many other parts of VuFind®, so they can be easily created and extended. Review Questions ---------------- 1. How does the router impact which controller action gets called? 2. Why would a controller action return an HTTP response object instead of a view model? -3. What class do most VuFind controllers extend from? +3. What class do most VuFind® controllers extend from? diff --git a/source/conf.py b/source/conf.py index 3e27b47..103dfd4 100644 --- a/source/conf.py +++ b/source/conf.py @@ -57,7 +57,7 @@ master_doc = 'index' # General information about the project. -project = u'Learning VuFind' +project = u'Learning VuFind®' copyright = u'2020, Demian Katz - This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License' author = u'Demian Katz' @@ -66,9 +66,9 @@ # built documents. # # The short X.Y version. -version = u'1.1.0' +version = u'1.1.1' # The full version, including alpha/beta/rc tags. -release = u'v1.1.0' +release = u'v1.1.1' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. @@ -142,7 +142,7 @@ # The name for this set of Sphinx documents. # " v documentation" by default. # -# html_title = u'Learning VuFind' +# html_title = u'Learning VuFind®' # A shorter title for the navigation bar. Default is the same as html_title. # @@ -279,7 +279,7 @@ # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ - (master_doc, 'LearningVuFind.tex', u'Learning VuFind', + (master_doc, 'LearningVuFind.tex', u'Learning VuFind®', u'Demian Katz', 'manual'), ] @@ -321,7 +321,7 @@ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ - (master_doc, 'learningvufind', u'Learning VuFind', + (master_doc, 'learningvufind', u'Learning VuFind®', [author], 1) ] @@ -336,8 +336,8 @@ # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ - (master_doc, 'LearningVuFind', u'Learning VuFind', - author, 'LearningVuFind', 'This book provides basic insights for installation, configuration and customisation of the Open Source Discovery Software VuFind.', + (master_doc, 'LearningVuFind', u'Learning VuFind®', + author, 'LearningVuFind', 'This book provides a basic introduction to installation, configuration and customization of the open source discovery software, VuFind®.', 'Miscellaneous'), ] diff --git a/source/index.rst b/source/index.rst index e229f1f..53dcb25 100644 --- a/source/index.rst +++ b/source/index.rst @@ -3,16 +3,16 @@ You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -Welcome to Learning VuFind! -=========================== +Welcome to Learning VuFind®! +============================ .. toctree:: :maxdepth: 2 00-preface -Part 1. VuFind Basics -_____________________ +Part 1. VuFind® Basics +______________________ .. toctree:: :maxdepth: 2 @@ -61,8 +61,8 @@ ________________________________________ 14-recommendation_modules 15-other_search_backends -Part 6. Extending and Customizing VuFind’s Code -_______________________________________________ +Part 6. Extending and Customizing VuFind®’s Code +________________________________________________ .. toctree:: :maxdepth: 2