RSS feed

Hi,

I'm wandering if it is possible to activate a rss feed in the forums. I think that would provide a nice way for some people (me included) to keep up with it.

Pedro Kroger

I'll look into it.

I'd like to put in a vote for RSS. I use it for many fora and blogs. It's a great time-saver. thanks

I agree!

Add my vote.

And another vote - thanks! (And thanks, of course, for setting up this forum!)

I found these rss plugins for phpbb:

https://tools.wackomenace.co.uk/phpbb-rss/
http://www.phpbb.com/community/viewtopi ... 0&t=544331

pedro

Is there yet some RSS-Feed with the new topics?

schoppenhauer wrote:Is there yet some RSS-Feed with the new topics?

Nope, not yet. I just need to find the time and the energy to sit down and get it done. Apologies to everybody waiting for this. It's on my list.

findinglisp wrote:

schoppenhauer wrote:Is there yet some RSS-Feed with the new topics?

Nope, not yet. I just need to find the time and the energy to sit down and get it done. Apologies to everybody waiting for this. It's on my list.

BTW, I'm traveling right now and most of this week, so I'll try to look at it next weekend.

Hi, i also could benefit from a simple RSS feed so i leave some more rss plugins i found on phpBB's mod site:

RSS Feed 2.0
RSS Syndication mod
Simple RSS mod for phpBB3
Simple syndication
Smartfeed

now it's a matter of finding the most convenient one, i will leave that to you guys. I will, however, suggest that if anyone has had any experience with any of these mods to go ahead and share it.

AlexPaes wrote:now it's a matter of finding the most convenient one, i will leave that to you guys. I will, however, suggest that if anyone has had any experience with any of these mods to go ahead and share it.

Thanks, Alex. That's helpful. I had done some research a couple of months ago, too, but the main thing stopping me from moving forward is that I found three or four different packages without having any one of them be a clear winner. The problem, I'm finding, with phpBB mods is that you basically have to hack them into the phpBB source at various different levels, which makes it a royal pain to try three or four of them to see which you like best.

Thus, if anybody has any experience with this stuff, as Alex said, it would be very helpful for me so that I don't spend any time trying out something that's a known dud.

Well, I really want to have rss on lispforum (I always miss discussions because I don't check the webpage often). How can I help you with that? I can install phpBB here, try the different plugins and send you a recipe of how to install the best one. Do you think that would be helpful?

kroger wrote:Well, I really want to have rss on lispforum (I always miss discussions because I don't check the webpage often). How can I help you with that? I can install phpBB here, try the different plugins and send you a recipe of how to install the best one. Do you think that would be helpful?

Yea, that would actually be helpful. Unfortunately, with job and family activities, I'm just not having enough time to do the research. Every RSS package that I have looked at involves some amount of hacking the phpBB source to install, so I was sort of put off by that initially. There are also quite a few different ones that work more or less well, some that provide granularity at the individual forum level and some at the site level. I wanted something that would provide both site and forum level granularity, so somebody who was interested in Scheme only, say, could subscribe to just the Scheme forum feed, while somebody interested in the complete site could subscribe to the top-level. Anyway, if you have the time and the initiative to help, I would be grateful.

Just for fun I hacked a little script that parses the "active posts" (which should contain all new ones I think) and puts them into some RSS-Feed (hopefully - actually I didnt try it out yet, but if it doesnt validate, it should be easy to adapt):

(defun http-page-to-string (path)
  (let*
      ((data (make-array 65535
			 :adjustable nil))
       (response (trivial-http:http-get path))
       (stream (caddr response))
       (rd 0)
       (ret ""))
    (tagbody
       start
       (setf rd (read-sequence data stream))
       (setf ret (concatenate 'string
			      ret
			      (coerce (subseq data 0 rd) 'string)))
       (if (= rd 65535) (go start)))
    ret))

(defun get-entries (path)
  (prog ((scanner (cl-ppcre:create-scanner "<a href=\"([^\"]*)\" class=\"topictitle\">([^<]*)</a>"))
	(page (http-page-to-string path))
	(start 0)
	(ret nil))
       scan-it
       (multiple-value-bind
	     (regstart end groupstarts groupends)       
	   (cl-ppcre:scan scanner page :start start)
	 (cond (regstart
		(push (cons
		       (subseq page (elt groupstarts 0) (elt groupends 0))
		       (subseq page (elt groupstarts 1) (elt groupends 1)))
		      ret)
		(setf start end)
		(go scan-it))
	       (t (return (nreverse ret)))))))

(defun output-rss (entries)
  (with-output-to-string (s)
    (format s "<?xml version=\"1.0\" encoding=\"utf-8\"?>
<rss version=\"2.0\">
<channel>
    <title>lispforum</title>
    <link>http://www.lispforum.com/</link>
    <description>Lisp Forum active Posts</description>
    <image>
      <url>http://www.lispforum.com/styles/prosilver/imageset/lispforum-logo4.png</url>
      <title>LispForum</title>
      <link>http://www.lispforum.com/index.php</link>
    </image>")
    (let
	((scanner (cl-ppcre:create-scanner "t=([0-9]*)&")))
      (dolist (entry entries)
	(format s "<item><title>~a</title><link>~a</link><guid>~a</guid></item>"
		(cdr entry) (car entry)
		(multiple-value-bind
		      (st en grst gren)
		    (cl-ppcre:scan scanner (car entry))
		  (declare (ignore st en))
		  (subseq (car entry) (elt grst 0) (elt gren 0))))))
    (format s "</channel></rss>")))
 


(defun get-rss ()
  (output-rss (get-entries "http://www.lispforum.com/search.php?search_id=active_topics")))

It uses trivial-http and ppcre. Example:

CL-USER> (get-rss)

"<?xml version=\"1.0\" encoding=\"utf-8\"?>
<rss version=\"2.0\">
<channel>
    <title>lispforum</title>
    <link>http://www.lispforum.com/</link>
    <description>Lisp Forum active Posts</description>
    <image>
      <url>http://www.lispforum.com/styles/prosilver/imageset/lispforum-logo4.png</url>
      <title>LispForum</title>
      <link>http://www.lispforum.com/index.php</link>
    </image><item><title>How Do You Teach Yourself Programming?</title><link>./viewtopic.php?f=20&t=157&sid=b0631e85f8b000488e3c069375992b68</link><guid>157</guid></item><item><title>Copying a multidimensional array</title><link>./viewtopic.php?f=2&t=383&sid=b0631e85f8b000488e3c069375992b68</link><guid>383</guid></item><item><title>Coercing to double-float</title><link>./viewtopic.php?f=2&t=384&sid=b0631e85f8b000488e3c069375992b68</link><guid>384</guid></item><item><title>Lisp Quiz #3</title><link>./viewtopic.php?f=32&t=381&sid=b0631e85f8b000488e3c069375992b68</link><guid>381</guid></item><item><title>Has anyone yet used Liskell?</title><link>./viewtopic.php?f=29&t=385&sid=b0631e85f8b000488e3c069375992b68</link><guid>385</guid></item><item><title>Which HTTP Server?</title><link>./viewtopic.php?f=2&t=376&sid=b0631e85f8b000488e3c069375992b68</link><guid>376</guid></item><item><title>sbcl + aserve</title><link>./viewtopic.php?f=2&t=375&sid=b0631e85f8b000488e3c069375992b68</link><guid>375</guid></item><item><title>Embedding PHP into Common Lisp</title><link>./viewtopic.php?f=2&t=382&sid=b0631e85f8b000488e3c069375992b68</link><guid>382</guid></item><item><title>Lisp and GPU computing</title><link>./viewtopic.php?f=2&t=380&sid=b0631e85f8b000488e3c069375992b68</link><guid>380</guid></item><item><title>Extending a simple array dynamically</title><link>./viewtopic.php?f=2&t=377&sid=b0631e85f8b000488e3c069375992b68</link><guid>377</guid></item><item><title>write-line</title><link>./viewtopic.php?f=29&t=378&sid=b0631e85f8b000488e3c069375992b68</link><guid>378</guid></item></channel></rss>"
CL-USER> 

In the links it also grabs session-id's (which could be replaced easily by additional regexing), as guid it uses the t-parameters of the url's.

Some quick comments/questions:

- You actually use TAGBODY, PROG and GO in high level code?
- Reading a page into a string should be possible much easier.
- HTML is not a regular language. Using regular expressions to parse it can only be a brittle hack. There are some very nice XML/HTML parser libraries available for CL.

Harleqin wrote:You actually use TAGBODY, PROG and GO in high level code?

Unfortunately using "do", I couldnt find a possibility to do something until some condition holds (i.e. do it when it holds once again). And I dont like loop. And it is no high-level-code, because it uses the lowlevel read-sequence-stuff.[/quote]

Harleqin wrote:Reading a page into a string should be possible much easier.

I never used CL for HTTP-Access so far. This possibility seemed sufficient, but since I was sure that its possible easier, I put this part into an own function - feel free to modify it.

Harleqin wrote:HTML is not a regular language. Using regular expressions to parse it can only be a brittle hack.

Actually, downloading this page to generate an RSS-Feed from it is a hack itself. If I wanted to have something "clean", I would have gone down to the databases. But well, its a hack, but it works well.

Harleqin wrote:There are some very nice XML/HTML parser libraries available for CL.

I have tried s-xml and xmls. Both do not understand the entity "•", and both dont finish to parse it. Also cxml has problems. I would like to have cl-dom, but ... I couldnt find a download-link. Actually, there are at least a dozen of XML-Libraries for Common Lisp out there, but none really working. I think the best possibility would be to just bind the official DOM-Parser (using some java-binding or so), should be rather trivial, but I think noone did this so far.

OK, a quick refactoring (untested):

(defun http-page-to-string (path)
  (with-open-stream (in-stream (third (trivial-http:http-get path)))
    (with-output-to-string (out-stream)
      (loop
        (let ((line (read-line in-stream nil)))
          (when (null line) (return))
          (write-line line out-stream)))))

Note that this uses the simple LOOP.

Regarding the HTML parsing: I haven't done this before, but I thought of Closure HTML.

Harleqin wrote:Regarding the HTML parsing: I haven't done this before, but I thought of Closure HTML.

Try it (Its possible, but annoying)

Harleqin wrote: - Reading a page into a string should be possible much easier.

The easy way is to use DRAKMA.

(drakma:http-request uri)

will return you a string (http-request has _a lot_ of options, it can return not just strings).

Maybe another interesting question would be whether the thing it basically does is usefull to provide an RSS-Feed for this Forum (no matter how exactly it is implemented now).

schoppenhauer wrote:

Harleqin wrote:Regarding the HTML parsing: I haven't done this before, but I thought of Closure HTML.

Try it (Its possible, but annoying)

This should work, but I can't test it currently:

(defun get-entries (path)
  (let ((page (chtml:parse (trivial-http:http-get path) (stp:make-builder)))
        (entries ()))
    (stp:do-recursively (el page)
      (when (and (typep el 'stp:element)
                 (equal (stp:local-name el) "a")
                 (equal (stp:attribute-value el "class") "topictitle"))
        (push (cons (stp:attribute-value el "href")
                    (stp:string-value el))
              entries)))))

Note that it parses the stream directly, and doesn't need the html-page-to-string function. It uses Closure HTML and CXML-STP (a nice alternative for DOM).

schoppenhauer wrote:Maybe another interesting question would be whether the thing it basically does is usefull to provide an RSS-Feed for this Forum (no matter how exactly it is implemented now).

Yes, absolutely.

Just so we're clear... LispForum runs on phpBB on a shared hosting account. I have no ability to run any CL on that system, so it really doesn't matter in terms of adding RSS capabilities to the forums. There are several RSS systems for phpBB, but phpBB being written in PHP, they all require you to actually patch the phpBB source code directly. I simply haven't had the time to try them all out. Kroger offered to do some research to come up with the best RSS implementation and then I'll have to go in and patch things.

Just +1 on wanting RSS. Tbh that this forum system doesn't support it defaultly seems little pathetic... I made an(admittedly untested) rss writer for 'my' project, and it isn't that hard..

Not to diss anyone except for phpBB board :p

Jasper wrote:Just +1 on wanting RSS. Tbh that this forum system doesn't support it defaultly seems little pathetic... I made an(admittedly untested) rss writer for 'my' project, and it isn't that hard..

Not to diss anyone except for phpBB board :p

Yes, agreed. I would thought that having RSS embedded into every web application these days would be the default, but alas it is not.