RSS reader symfony 1.4 tutorial | Welcome to SymfonyLab

3D render of an RSS symbol

Thanks to symfony hosting ServerGroove we’ve got eventually a chance to implement our old idea about super-duper RSS reader which would work as web service for which we can add multiple features on demand (yes there is still no good enough web RSS reader for us).

So lets see how to write RSS reader on symfony in 15 minutes (we’ve already had some basics written for symfony 1.2 with propel so it was good chance to figured out doctrine differences). Here goes main steps we did:

1. symfony generate:project rssreader
2. symfony generate:app frontend
3. create DB called rssreader
4. create doctrine/schema.yml
5. symfony doctrine:build –all –no-confirmation –and-load
6. symfony plugin:install sfFeed2Plugin
7. symfony plugin:install sfWebBrowser

For schema file we propose to use this one:

category:
  actAs: { Timestampable: ~ }
  columns:
    name:         { type: string(255) }
    description:  { type: string(4000) }
    parent_id:    { type: integer, default: 0 }
 
source:
  actAs: { Timestampable: ~ }
  columns:
    category_id:  { type: integer, notnull: true }
    name:         { type: string(255) }
    description:  { type: string(4000) }
    link:         { type: string(4000) }
    author:       { type: string(255) }
    pub_date:     { type: varchar(100) }
    generator:    { type: varchar(100) }
    language:     { type: varchar(100) }
    response:     { type: string(4000) }
    notes:        { type: string(4000) }
    is_private:   { type: boolean }
  relations:
    category: { onDelete: CASCADE, local: category_id, foreign: id, foreignAlias: Sources }
 
feed:
  actAs: { Timestampable: ~ }
  columns:
    source_id:    { type: integer, notnull: true }
    name:         { type: string(255) }
    link:         { type: string(4000) }
    summary:      { type: string(4000) }
    description:  { type: string(4000) }
    author:       { type: string(255) }
    response:     { type: string(4000) }
    notes:        { type: string(4000) }
  relations:
    source: { onDelete: CASCADE, local: source_id, foreign: id, foreignAlias: Feeds }

So we have category / feed source and actual feeds tables.
At this point worth to mention that we had to convert Propel schema to Doctrine, e.g. replace type names:

varchar(255) => string(255)
longvarchar => string(4000)

Also when we tried to install sfFeed2 and sfWebBrowser plugins it said:

Unable to get plugin licence information for plugin "sfFeed2Plugin": Unknown package: "sfFeed2Plugin" (Debug: File http://plugins.symfony-project.org:80/REST/sffeed2plugin/info.xml not valid (received: HTTP/1.0 404 No version available with the installed symfony version)) (use --force-license to force installation)

We did not spend time on figuring out the reason of it (as our goal is to have rss reader in 15 minutes) so we simply downloaded plugins as tgz and unarchived them into lib directory.

Now important part of project is actual rss reader script. It’s pretty straightforward though, we put it into “batch” directory so we can execute it as cron task:

 
  require_once(dirname(__FILE__) . '/../config/ProjectConfiguration.class.php');
  $configuration = ProjectConfiguration::getApplicationConfiguration('frontend', 'dev', true);
  sfContext::createInstance($configuration);
 
  $databaseManager = new sfDatabaseManager($configuration);
  $databaseManager->loadConfiguration();
 
  $sources = Doctrine::getTable('Source')->createQuery('a')->execute();
 
  foreach ($sources as $source) {
 
    $feeds = sfFeedPeer::createFromWeb($source->getLink());
    foreach ($feeds->getItems() as $feed) {
 
      $q = Doctrine_Query::create()
        ->select('COUNT(id) as cnt')
          ->from('feed')
          ->where('link = ?', $feed->getLink());
      $results = $q->execute();
      if ($results[0]['cnt']) continue;
 
      $d = new Feed();
      $d->setSourceId($source->getId());
      $d->setName($feed->getTitle());
      $d->setLink($feed->getLink());
      $d->setDescription($feed->getDescription());
      $d->save();
 
    }
 
  }

So it just goes through sources table and retrieve new unique feeds. Lets run it daily.
Lets also skip backend part for now (of course we could quickly make it with backend generator but most probably you would like to import your favourite feed sources as CSV or so). So you only have to add somehow rows into source table with proper feed link (“link” column is for that)

To read feeds, lets just create home module with actions.class.php:

 
class homeActions extends sfActions
{
  public function executeIndex(sfWebRequest $request)
  {
    $this->sources = Doctrine::getTable('Source')
      ->createQuery('a')
      ->execute();
 
    $this->feeds = Doctrine::getTable('Feed')
      ->createQuery('a')
      ->execute();
  }
 
}

And indexSuccess.php template:

<h1>Recently Added RSS Feeds</h1>
 
<table width="100%">
  <tbody>
    <?php foreach ($feeds as $feed): ?>
    <tr>
      <td><?php echo link_to($feed->getName(), $feed->getLink(), "target='_blank' id='feed{$feed->getId()}'") ?></td>
      <td width="1%"><?php echo link_to(" ", $feed->getSource()->getLink(), 'target="_blank" class="feed"') ?></td>
    </tr>
   <?php endforeach; ?>
  </tbody>
</table>
 
 
<h1>Recently Added RSS Sources</h1>
 
<table width="100%">
  <tbody>
    <?php foreach ($sources as $source): ?>
    <tr>
      <td><?php echo $source->getCategory() ?></td>
      <td><?php echo link_to($source->getName(), $source->getLink()) ?></td>
      <td><?php echo $source->getAuthor() ?></td>
      <td><?php echo $source->getPubDate() ?></td>
      <td width="1%"><?php echo link_to(" ", $source->getLink(), 'target="_blank" class="feed"') ?></td>
    </tr>
    <?php endforeach; ?>
  </tbody>
</table>

Don’t forget to change config/routing.yml so it has:

1
2
3

homepage:
  url:   /
  param: { module: home, action: index }

That’s all. Cookie is ready to eat. Run feeds fetching script from batch and make sure you see feeds on homepage of your project.

Very soon we are going to deploy this project on our ServerGroove account and you can give it a try.

Also we are planning to keep it as open-source ongoing project.

By the way, how do you think what is the best way to provide ability for other developers to create own plugins for it? Which API may fit this project?

4 replies on “RSS reader symfony 1.4 tutorial”

I do not like the batch approach. Batch scripts are dead since mmmh symfony 1.2 i think?
So first of all, this batch logic should be moved into a task.
BUT i do not like it there too. I do not like a system which depends on cron scripts. Imagine those scripts are not running due to some bug/problem, then the webseite will show either outdated data or event nothing.
I would fetch the rss feeds by runtime and add a caching layer between it. In this case you only do stuff, which is requested and the system does not depend to “external” scripts.

This is great point though 🙂

I would not read feeds by runtime b/c of a few reasons

1. you will lose feeds when you wont login to system for some time
2. where there are lot of feeds – it takes time before it all loads
etc

As for the tasks – thanks for the great tip I almost forgot about this feature.

I agree with you than cron is not enough reliable but honestly not sure what else can be used.

My point – does not really matter if this batch or task – main thing is make sure that cron started fetching and completed it succesfully (so need some kind of processes manager)

[…] continue to our previous post RSS Reader Symfony 1.4 Tutorial we’ve decided to add to our RSS Reader search feature. One which may fits our purposes is […]

[…] where job queue comes to help you. In specific jobs queue is something we have to use in our symfony-driven RSS reader – we want to make sure parsing of each feed source can be done independently and in […]

4 replies on “RSS reader symfony 1.4 tutorial”

Leave a Reply