atom feed8 messages in net.php.lists.pear-devRe: [REST] making list-upgrades faster
FromSent OnAttachments
TiasMar 26, 2007 3:24 pm 
Igor FeghaliMar 26, 2007 4:15 pm 
David CoallierMar 26, 2007 4:24 pm 
Christian WeiskeMar 26, 2007 10:11 pm 
David CoallierMar 26, 2007 10:19 pm 
Gregory BeaverMar 26, 2007 10:31 pm 
David CoallierMar 26, 2007 10:37 pm 
Gregory BeaverMar 27, 2007 9:16 am 
Subject:Re: [REST] making list-upgrades faster
From:Gregory Beaver (
Date:Mar 27, 2007 9:16:48 am


This is not going to be forgotten. FYI:

I've added a roadmap TODO item for this request. Thanks for your understanding and patience.


Tias wrote:


This is a proposition for a new XML file for the REST server, which can speed up the 'list-upgrades' command significantly.

First off, I like REST: It's a conceptually simple idea and it works great. The only current disavdvantage is that some PEAR commands need to open a lot of different XML-files, which makes it slow.

Let's investigate the 'list-upgrades' command, which shows if there is an upgrade available for a package you have installed. In REST/10.php::listLatestUpgrades() p/packages.xml is fetched (~10k) for every package you have installed: r/<package>/allreleases.xml is fetched (~1k) r/<package/<state>.xml is fetched (poss. cached) (~5B) done

A minimal amount of data is downloaded, _but_ the amount of files opened grows with the amount of packages installed. If you have 4 packages installed (everybody has) you download up to 9 files each time(~14k). If you have 15 packages installed, you download up to 31 files each time(~25k) ! I will presume you know the cost of opening/closing connections and files, it is significant, and certainly in todays broadband times.

The listLatestUpgrades() command uses only the package names, and the different states with its latest version of these packages. For one server, this information could be gathered in one file, eg. latestpackages.xml <!-- example xml file with packages and latest version info --> <!-- full descriptive names used for clarity --> <channel></channel> <package> <name>Example_Tias</name> <latest> <row><state>stable</state><version>1.3.2</version></row> <row><state>beta</state><version>1.4.0a2</version></row> </latest> </package> <package> ...

This file would contain for every package its name, and every 'highest' state which has a different version number. The latest is needed for state-friendly package management: only the highest state and its version are shown, if a lower state has a higher version than that one, it is shown too.

This file would be at most 3 times as big as the current allpackages.xml file (thus ~30k), yet listLatestUpgrades() would only need 1 file to download: a significant speedup.

If there are any questions, do ask. Greetings, Tias