Continuous updates, more books

01:46:14 18-Feb-2004

I've been working on the incremental updates to the search indexes that are used when you search for book titles, authors or publishers. Inability to update these indexes is the reason why there were no new books since mid-november 2003.

Now that the new index is in place, the books are updated non-stop -- re-load the front page and chances are you'll see new books every couple of minutes. Indexes do have a delay though, normally up to 24 hours -- so, recently loaded books can be browsed into, but can't be found by words in title.

There is still some optimisation work to do on the searching counterpart, but overall I'm satisfied with the results.

Accumulated during this time raw data represents about 1 million books -- that should keep the loader busy for a while. As of the time of this writing there are about 440,000 books in the database, roughly I expect the loader to add something between 1000 and 5000 books a day.

This time the book with entries in most libraries is "Liberty in America, 1600 to the present" -- with 8 libraries listing it. Last time the winning book was "Ethics in the world of business" -- I can't wait to see what the winning title will be in a month.

--
Andrew.

P.S.
This bit of news is somewhat bloggish/personal in appearance, sorry if it bothers you.