- 1 What is Global search?
- 2 What can I search for?
- 3 How does it work?
- 4 How is it set up?
- 4.1 Setting up Solr
- 4.2 How to install Solr
- 4.2.1 Linux (Debian/Ubuntu)
- 4.2.2 Linux (Redhat/Centos 6 & 7)
- 4.2.3 OSX using macports
- 4.2.4 OSX using homebrew
- 4.2.5 Windows
- 4.2.6 The Solr server
- 188.8.131.52 Solr 5/6 schema setup
- 184.108.40.206 Solr 4 schema setup
- 220.127.116.11 Memory and File indexing considerations
- 18.104.22.168 Too many boolean clauses error
- 4.2.7 Commannd for installing Solr Server on Centos 7
- 5 See also
What is Global search?
- Global search lets you search everywhere on the Moodle site that you have access to. A student can search their courses for particular lecture notes, for example, or a teacher could search for subject-related activities.
- The feature needs to be enabled by the administrator and a search box is then available next to the user menu.
- A Global search block may also be added to course pages.
What can I search for?
We can search for our courses, activities information and some activities contents like forum posts, book chapters, glossary entries or collaborative wikis pages. We will add more contents in future Moodle versions. Developers interested in adding plugin's contents to global search may be interested in looking at the Search API.
How does it work?
- Click the search icon by the user menu and type a search term into the box that appears, or type into the box in the Global search block if enabled.
- On the next screen, we can simply click the search button to search everywhere, or expand the Filter to search in specific areas:
- We will then see results displayed from all areas of Moodle you have access to:
Solr search query features
We can improve your search using any of the following search query features:
- Specifying the field to be searched by prefixing the search query with 'title:', 'content:', 'name:', or 'intro:' e.g.'title:news' returns results with the word 'news' in the title
- Boolean operators ('AND', 'OR', 'NOT') to combine or exclude keywords
- Wildcard characters ('*' or '?' ) to represent characters in the search query
- Proximity searches ('~') e.g. mood~2 returns "moodle" (2 letters away from "mood"), Moodle Australia~3 returns results containing "Moodle HQ in Perth, Australia" (the queried terms were within 3 words of each other)
- Boosting terms ('^') to boost certain words or phrases e.g. "Perth Australia"^5 "Australia" will return results with the phrase "Perth Australia" as more relevant.
Note: The above features are only available when using the Solr search engine.
How is it set up?
Note: Global search needs a search engine. Instructions for the installing the PHP Solr extension and a Solr server are below. See the Developer docs on Search engine plugins if you wish to write your own. It's recommended you set everything up first, index the site contents and only then enable Global search.
Setting up Solr
- Set the feature up in Site administration > Plugins > Search > Manage global search by selecting Solr as the search engine and ticking search area checkboxes as required
- In Site administration > Plugins > Search > Solr, set Host name to localhost, Port to 8983 and Index name to 'moodle' (the name of the index in Solr)
- If we are using Solr with SSL encryption, we will need to configure Moodle as such. This is important if solr is not installed in the web server server.
- We will need a separate key file and cacert file, both in pem format, located on your server Moodle, and readable by the PHP process.
- Go to Site administration > Plugins > Search > Solr
- Set Secure mode to Yes
- SSL certificate to /path/to/certs/solr-ssl.cacert.pem
- SSL key to /path/to/certs/solr-ssl.key.pem
- SSL key Password to The password used to lock the SSL Key
- SSL CA certificates name to /path/to/certs/solr-ssl.cacert.pem
- You now need to populate the created Solr index with your site's data. We can do it via the web interface by going to Site administration > Reports > Global search info or from the CLI by running the search/cli/indexer.php script. The CLI script is the recommended option for big sites.
# sudo -u www-run php search/cli/indexer.php --force
- Enable Global search in Site administration > Advanced features
Solr has the ability to index the contents of files, such as File resources and attachments to Forum posts. This uses the Tika engine which comes bundled with Solr. To enable this feature:
- In Site administration > Plugins > Search > Solr enable the checkbox Enable file indexing
- Set Maximum file size to index (kB) to some value - the default is 2097152 (2GB)
- Files larger than this limit will not be sent for Solr for indexing, but the file name will still be indexed.
How to install Solr
You need PHP Solr extension installed. We can download the official latest versions from (http://pecl.php.net/package/solr) The minimum required version is PECL Solr 2.1 for PHP 5 branch and PECL Solr 2.4 for PHP 7 branch.
Basic installation steps (using apache web server):
With PHP5.xsudo apt-get install libpcre3-dev libxml2-dev libcurl4-openssl-dev sudo apt-get install php5-dev sudo apt-get install php-pear sudo pecl install solr sudo sh -c "echo 'extension=solr.so' > /etc/php5/apache2/conf.d/solr.ini" sudo sh -c "echo 'extension=solr.so' > /etc/php5/cli/conf.d/solr.ini" sudo service apache2 restart
With PHP 7sudo apt-get install libpcre3-dev libxml2-dev libcurl4-openssl-dev sudo apt-get install php7.0-dev sudo apt-get install php-pear sudo pecl install solr sudo sh -c "echo 'extension=solr.so' > /etc/php/7.0/apache2/conf.d/solr.ini" sudo sh -c "echo 'extension=solr.so' > /etc/php/7.0/cli/conf.d/solr.ini" sudo service apache2 restart
Linux (Redhat/Centos 6 & 7)
Using built in php5 packagesyum install php-pecl-solr2 service httpd restart
Using 3rd party php7 packages (webtactic)yum install libxml2-devel pcre-devel libcurl-devel php70w-devel php70w-pear pecl install solr echo "extension=solr.so" > /etc/php.d/solr.ini service httpd restart
OSX using macportssudo port install apache-solr4 sudo port install php54-solr
OSX using homebrewbrew install homebrew/php/php56-solr
Install the pecl package as usual. (This has not yet been tested.)
The Solr server
Moodle 3.1 supports Solr server from 4.0 onwards, although we can only use the Solr schema setup script that we provide with Moodle from Solr 5. The latest Solr 5 available version is the recommended one; the same will apply to Solr 6 once it is released.
Note that for medium/large sites you may need to increase maxBooleanClauses setting. In [http://tracker.moodle.org/browse/MDL-54992 MDL-54992] we are working on an alternative way to query the server.
The following example snippet (feel free to copy & paste into a .sh script with execution permissions) will download Solr 5.4.1 (replace it with latest 5.x) in the current directory, start the solr server and create an index in it named moodle to add moodle data to it. If wget gives an error, check http://www-us.apache.org/dist/lucene/solr and update SOLRVERSION#!/bin/bash set -e SOLRVERSION=5.4.1 SOLRNAME=solr-$SOLRVERSION SOLRTAR=$SOLRNAME'.tgz' INDEXNAME=moodle if [ -d $SOLRNAME ]; then echo "Error: Directory $SOLRNAME already exists, remove it before starting the setup again." exit 1 fi if [ ! -f $SOLRTAR ]; then wget http://www-us.apache.org/dist/lucene/solr/$SOLRVERSION/$SOLRTAR fi tar -xvzf $SOLRTAR cd $SOLRNAME bin/solr start bin/solr create -c $INDEXNAME # After setting it up and creating the index use: # - "/yourdirectory/solrdir/bin/solr start" from CLI to start the server # - "/yourdirectory/solrdir/bin/solr stop" from CLI to stop the server.
Solr 5/6 schema setup
Moodle will use Solr's managed schema interface to install the required fields. We will be directed on what to do from the Manage global search page.
For very large or busy sites, it is recommended that you manually remove the _text_ field, and associated copy directive from, the default Solr schema. This field is not used by Moodle, and will significantly slow indexing, and increase the size of the resulting Solr core.
Solr 4 schema setup
We cannot use the schema setup script when using a Solr 4 server. If you really want to use the Solr 4x branch, here are the field types descriptions:
Extracted from search/classes/document.php
Field name Field type Stored Indexed Query field id org.apache.solr.schema.StrField true false false itemid org.apache.solr.schema.TrieIntField true true false title org.apache.solr.schema.TextField true true true content org.apache.solr.schema.TextField true true true contextid org.apache.solr.schema.TrieIntField true true false areaid org.apache.solr.schema.StrField true true false type org.apache.solr.schema.TrieIntField true true false courseid org.apache.solr.schema.TrieIntField true true false owneruserid org.apache.solr.schema.TrieIntField true true false modified org.apache.solr.schema.TrieDateField true true false userid org.apache.solr.schema.TrieIntField true true false description1 org.apache.solr.schema.TextField true true true description2 org.apache.solr.schema.TextField true true true solr_filegroupingid org.apache.solr.schema.StrField true true false solr_fileid org.apache.solr.schema.StrField true true false solr_filecontenthash org.apache.solr.schema.StrField true true false solr_fileindexstatus org.apache.solr.schema.TrieIntField true true false solr_filecontent org.apache.solr.schema.TextField false true true
Memory and File indexing considerations
For large sites, and particularly if we are using the File indexing feature, it's a good idea to give Solr plenty of memory, eg aound 10-20GB. To start Solr with more than its default 512MB of RAM, use the -m option:solr start -m 12g
See the documents for your version of Solr on how to increases memory when Solr is started automatically.
Too many boolean clauses error
Because of the way Moodle handles permissions for searches, if you have non-admin users with access to a large number of activities (>1000), they may an error similar to the following:The query you provided could not be parsed by the search engine: org.apache.solr.search.SyntaxError: Cannot parse ... too many boolean clauses
To correct this, you need to increase the maxBooleanClauses setting in your Solr core. The setting is located in corename/conf/solrconfig.xml.
This important note is included in the config file:** WARNING ** This option actually modifies a global Lucene property that will affect all SolrCores. If multiple solrconfig.xml files disagree on this property, the value at any given moment will be based on the last SolrCore to be initialized.
This means that for consistent behavior you should update this value for all cores in the Solr server.
Commannd for installing Solr Server on Centos 7
First of all install java (openjdk)yum install java-1.8.0-openjdk
Add user for solradduser solr
Download solr server, find latest here : http://lucene.apache.org/solr/mirrors-solr-latest-redir.html, from the list there download latest solr-[version].tgz
Here is the list of needed commands (note that latest solr is suggested, change command to appropriate version)cd /opt wget http://www-eu.apache.org/dist/lucene/solr/6.1.0/solr-6.1.0.tgz tar zxvf solr-6.1.0.tgz cp solr-6.1.0/bin/install_solr_service.sh . rm -rf solr-6.1.0 ./install_solr_service.sh solr-6.1.0.tgz chkconfig solr on
Create the indexsu - solr -c "/opt/solr/bin/solr create_core -c moodle"
Then go to your Moodle and set index name as created (moodle)