-
Notifications
You must be signed in to change notification settings - Fork 32
Home
Contents
====
- a. Introduction to acts_as_xapian
- b. Installation
- c. Comparison to acts_as_solr (as on 24 April 2008)
- d. Documentation – indexing
- e. Documentation – querying
a. Introduction to acts_as_xapian
=========
Xapian is a full text search engine library, which has Ruby bindings.
acts_as_xapian adds support for it to Rails. It is an alternative to
acts_as_lucene or acts_as_ferret.
Xapian is an offline indexing search library – only one process can have the
Xapian database open for writing at once, and others that try meanwhile are
unceremoniously kicked out. For this reason, acts_as_xapian does not support
immediate writing to the database when your models change.
Instead, there is a ActsAsXapianJob model which stores which models need
updating or deleting in the search index. A rake task ‘xapian:update_index’
then performs the updates since last change. Run it on a cron job, or similar.
Xapian 1.0.5 and associated Ruby bindings are required.
Email francis@mysociety.org with patches.
Installation===
Retrieve the plugin directly from the git version control system by running
this command within your Rails app.
c. Comparison to acts_as_solr (as on 24 April 2008)
=========
- Offline indexing only mode – which is a minus if you want changes
immediately reflected in the search index, and a plus if you were going to
have to implement your own offline indexing anyway.
- Collapsing – the equivalent of SQL’s “group by”. You can specify a field
to collapse on, and only the most relevant result from each value of that
field is returned. Along with a count of how many there are in total.
acts_as_solr doesn’t have this.
- No highlighting – Xapian can’t return you text highlighted with a search
query. You can try and make do with TextHelper::highlight (combined with
words_to_highlight below). I found the highlighting in acts_as_solr didn’t
really understand the query anyway.
- Date range searching – maybe this works in acts_as_solr, but I never found
out how.
- Spelling correction – “did you mean?” built in and just works.
- Multiple models – acts_as_xapian searches multiple models if you like,
returning them mixed up together by relevancy. This is like multi_solr_search,
only it is the default mode of operation and is properly supported.
- No daemons – However, if you have more than one web server, you’ll need to
work out how to use Xapian’s remote backend http://xapian.org/docs/remote.html.
- One layer – full-powered Xapian is called directly from the Ruby, without
Solr getting in the way whenever you want to use a new feature from Lucene.
- No Java – an advantage if you’re more used to working in the rest of the
open source world. acts_as_xapian, it’s pure Ruby and C++.
- Xapian’s awesome email list – the kids over at xapian-discuss are super
helpful. Useful if you need to extend and improve acts_as_xapian. The
Ruby bindings are mature and well maintained as part of Xapian.
http://lists.xapian.org/mailman/listinfo/xapian-discuss
d. Documentation – indexing
=======
1. Put acts_as_xapian in your models that need search indexing. For example:
acts_as_xapian :texts => [ :name, :short_name ], :values => [ [ :created_at, 0, “created_at”, :date ] ], :terms => [ [ :variety, ‘V’, “variety” ] ]Options must include:
- :texts, an array of fields for indexing with full text search. e.g.
:texts => [ :title, :body ] - :values, things which have a range of values for indexing, or for collapsing.
Specify an array quadruple of [ field, identifier, prefix, type ] where- number is an arbitary numeric identifier for use in the Xapian database
- prefix is the part to use in search queries that goes before the :
- type can be any of :string, :number or :date
:values => [ [ :created_at, 0, “created_at” ], [ :size, 1, “size”] ]
- :terms, things which come after a : in search queries. Specify an array
triple of [ field, char, prefix ] where- char is an arbitary single upper case char used in the Xapian database
- prefix is the part to use in search queries that goes before the :
:terms => [ [ :variety, ‘V’, “variety” ] ]
A ‘field’ is a symbol referring to either an attribute or a function which
returns the text, date or number to index. Both ‘number’ and ‘char’ must be
the same for the same prefix in different models.
Alternatively,
- :instead_index, a field which refers to another model that should be reindexed
instead of this one.
Options may include:
:eager_load, added as an :include clause when looking up search results in
database
:if, either an attribute or a function which if returns false means the
object isn’t indexed
2. Make and run this database migration to create the ActsAsXapianJob model.
class ActsAsXapianMigration < ActiveRecord::Migration def self.up create_table :acts_as_xapian_jobs do |t| t.column :model, :string, :null => false t.column :model_id, :integer, :null => false t.column :action, :string, :null => false end add_index :acts_as_xapian_jobs, [:model, :model_id], :unique => true end def self.down remove_table :acts_as_xapian_jobs end end3. Call ‘rake xapian::rebuild_index models=“ModelName1 ModelName2”’ to build the index
the first time (you must specify all your indexed models). It’s put in a
development/test/production dir in acts_as_xapian/xapiandbs.
4. Then from a cron job or a daemon, or by hand regularly!, call ‘rake xapian:update_index’
e. Documentation – querying
=======
If you just want to test indexing is working, you’ll find this rake task
useful (it has more options, see lib/tasks/xapian.rake)
rake xapian:query models=“PublicBody User” query=“moo”
To perform a query call ActsAsXapian::Search.new. This takes in turn:
model_classes – list of models to search, e.g. [PublicBody, InfoRequestEvent]
query_string – Google like syntax, see below
And then a hash of options:
:offset – Offset of first result
:limit – Number of results per page
:sort_by_prefix – Optionally, prefix of value to sort by, otherwise sort by relevance
:sort_by_ascending – Default true, set to false for descending sort
:collapse_by_prefix – Optionally, prefix of value to collapse by (i.e. only return most relevant result from group)
Google like query syntax is as described in http://www.xapian.org/docs/queryparser.html
Queries can include prefix:value parts, according to what you indexed in the
acts_as_xapian part above. You can also say things like model:InfoRequestEvent
to constrain by model in more complex ways than the :model parameter, or
modelid:InfoRequestEvent-100 to only find one specific object.
Returns an ActsAsXapian::Search object. Useful methods are:
description – a techy one, to check how the query has been parsed
matches_estimated – a guesstimate at the total number of hits
spelling_correction – the corrected query string if there is a correction, otherwise nil
words_to_highlight – list of words for you to highlight, perhaps with TextHelper::highlight
results – an array of hashes containing:
:model – your Rails model, this is what you most want!
:weight – relevancy measure
:percent – the weight as a %, 0 meaning the item did not match the query at all
:collapse_count – number of results with the same prefix, if you specified collapse_by_prefix
For more details about anything, see source code in lib/acts_as_xapian.rb -
please though do patch this file if there is documentation missing / wrong.