Tag: sphinx

Categories
symfony

Using Symfony and Sphinx

In continue to our previous post RSS Reader Symfony 1.4 Tutorial we’ve decided to add to our RSS Reader search feature. One which may fits our purposes is Sphinx.

So here are a few simple steps to prepare Sphinx for using

1. Get the latest stable release from here and untar it somewhere on your server
2. ./configure
3. make
4. make install
5. cp /usr/local/etc/sphinx.conf.dist /usr/local/etc/sphinx.conf (folder location may very slightly)
6. edit sphinx configuration by adding your DB details and search query:

vi /usr/local/etc/sphinx.conf

source src1
{
        type                                    = mysql
 
        sql_host                                = localhost
        sql_user                                = db_username
        sql_pass                                = db_password
        sql_db                                  = db_name
        sql_port                                = 3306  # optional, default is 3306
 
        sql_query                               = \
                SELECT id, source_id, created_at, name, description \
                FROM feed
 
        sql_attr_uint                   = source_id
        sql_attr_timestamp              = created_at
 
        sql_query_info                  = SELECT * FROM feed WHERE id=$id
}
 
 
index test1
{
        source                          = src1
        path                            = /var/data/test1
        docinfo                         = extern
        charset_type                    = sbcs
}
 
 
indexer
{
        mem_limit                               = 32M
}
 
 
searchd
{
        port                            = 9312
        log                             = /var/log/searchd.log
        query_log                       = /var/log/query.log
        read_timeout                    = 5
        max_children                    = 30
        pid_file                        = /var/log/searchd.pid
        max_matches                     = 1000
        seamless_rotate                 = 1
        preopen_indexes                 = 0
        unlink_old                      = 1
}

7. start indexing your database (the most probably you’ll need to run it periodically through cron):

/usr/local/bin/indexer test1

8. run actual Sphinx search daemon:

/usr/local/bin/searchd

When deamon is running there can be a problem with re-indexing:

using config file ‘/usr/local/etc/sphinx.conf’…
indexing index ‘test1’…
FATAL: failed to lock /var/data/test1.spl: Resource temporarily unavailable, will not index. Try –rotate option.

so you may need to stop it firstly (so kill search deamon, do indexing and start it again).

Thanks to ServerGroove symfony hosting we are able to setup it and investigate how it would work on their virtual server.

Now lets install sfSphinxPlugin to RSS Reader project and add some basic search code:

So first of all lets add search action to our actions.class.php:

public function executeSearch()
{
  $this->query = $this->getRequestParameter('q');
  $this->page = $this->getRequestParameter('p', 1);
  $options = array(
    'limit'   => 10,
    'offset'  => ($this->page - 1) * 10,
    'weights' => array(100, 1),
    'sort'    => sfSphinxClient::SPH_SORT_EXTENDED,
    'sortby'  => '@weight DESC',
    'port' => 9312
  );
  if (!empty($this->query))
  {
    $this->sphinx = new sfSphinxClient($options);
    $res = $this->sphinx->Query($this->query, 'test1');
    $this->pager = new sfSphinxDoctrinePager('Feed', $options['limit'], $this->sphinx);
    $this->pager->setPage($this->page);
    $this->pager->init();
    $this->logMessage('Sphinx search "' . $this->query . '" [' . $res['time'] .
                      's] found ' . $this->pager->getNbResults() . ' matches');
  }
}

So here worth to mention at least 2 important things:

We had to use:

1. parameter ‘port’ => 9312 to specify on which port Sphinx daemon is running.
2. sfSphinxDoctrinePager pager class which is luckily available in Sphinx plugin.

So all other here is pretty clear, just lets mention that we use index named test1 and we apply pager for table feed (mapped object Feed) which contains parsed RSS items.

Now lets create template searchSuccess.php which will show all found items and highlight searched keywords in them:

 
<?php use_helper('Search') ?>
 
<?php if (empty($query)): ?>
<?php return ?>
<?php endif ?>
 
<?php $res = $pager->getResults() ?>
<?php if (empty($res)): ?>
No result matches your query
<?php else: ?>
<?php if ($sphinx->getLastWarning()): ?>
Warning: <?php echo $sphinx->getLastWarning() ?>
<?php endif ?>
<ol start="<?php echo $pager->getFirstIndice() ?>">
<?php foreach ($res as $item): ?>
  <li>
    <h3><?php echo link_to(highlight_search_result($item->getName(), $query), $item->getLink(), 'target="_blank"') ?></h3>
    <?php echo html_entity_decode(highlight_search_result($item->getDescription(), $query)) ?>
  </li>
<?php endforeach ?>
</ol>
<?php endif ?>
 
<?php if ($pager->haveToPaginate()): ?>
  <?php echo link_to('&laquo;', '/home/search?q=' . $query . '&p=' . $pager->getFirstPage()) ?>
  <?php echo link_to('&lt;', '/home/search?q=' . $query . '&p=' . $pager->getPreviousPage()) ?>
  <?php $links = $pager->getLinks()?>
  <?php foreach ($links as $page): ?>
    <?php echo ($page == $pager->getPage()) ? $page : link_to($page, '/home/search?q=' . $query . '&p=' . $page) ?>
  <?php endforeach ?>
  <?php echo link_to('&gt;', '/home/search?q=' . $query . '&p=' . $pager->getNextPage()) ?>
  <?php echo link_to('&raquo;', '/home/search?q=' . $query . '&p=' . $pager->getLastPage()) ?>
<?php endif ?>

So for now we have ready RSS reader script with ability to search through the parsed feeds.
Next step we are going to implement some basic user interface which would allow for users to add own feed sources. This will be presented in our next article and after that we are going to make this project available for public collaborating at GitHub and run live version at ServerGroove symfony hosting.

Happy Christmas!