Re: Patch to re-order installation for dependencies

Hi, There are several other problems with this patch that can't be resolved until other areas are changed, so I'll withdraw it for now, unless someone has a bright idea on how to fix the original problem another way. Greg Greg Beaver wrote:

Hi, Those who have tried upgrade-all when you have many, many packages installed have noticed that it tends to die because the order of installation is important. The same is true of passing many package names on the command-line: the order is important. This patch seeks to take the first step towards solving that problem by grabbing dependency information from remote files (it is assumed that if the file is local, you can look up the dependency yourself, but there is no reason that can't be implemented as well). In addition, in many cases, you sit for a while downloading an application, only to find out that it depends on another one. I've patched in a call to remote-info that checks out dependencies, and fails prior to download if they aren't met. There is a design problem with the PEAR_Installer::install() method - it also downloads the packages, which means that to do any manipulation of installation order, it must be done in the PEAR_Command_Install::doInstall() method. I don't like the way it feels to hack in a patch there, so I wonder how hard it would be to move the downloading out of the install() method, and into a new download() method. This way, all of the packages could be downloaded and then ordered for installation. I think BC could be maintained by simply changing the doInstall() command to point to downloadAndInstall(), a wrapper that would call download and pass the downloaded file location to install(), which would operate unaware of any changes. There are potential issues, as I discovered today when pear.php.net was down. The XML_RPC class really doesn't like it when it can't connect, and will hang for a full two minutes before giving any error output. This means that without the code I added later, doing a "pear install package.xml" just hung the computer. Now, it does the same check that the installer does for downloadable packages, and as a hack, does a check to see if a package is a local package.xml by checking for package.xml in the name of the file. This is a big issue, and can be resolved by separating the download code from the install code, as then only local files need be passed to the install() command, and dependency checking of downloaded files can happen only in the download code. Another issue is I couldn't figure out is how to determine which version will be downloaded by an upgrade/install command, or whether I need to worry about it, because the remote-info just returns information about every release. In any case, try out the patch, with packages like SOAP that depend on HTTP_Request, which depends on other packages, etc. etc. Greg ------------------------------------------------------------------------ Index: pear/PEAR/Common.php =================================================================== RCS file: /repository/php-src/pear/PEAR/Common.php,v retrieving revision 1.97 diff -u -r1.97 Common.php --- pear/PEAR/Common.php 4 Aug 2003 13:34:25 -0000 1.97 +++ pear/PEAR/Common.php 4 Aug 2003 22:47:17 -0000 @@ -1678,6 +1678,88 @@

     }
      // }}}
+    // {{{ sortPkgDeps()

+

+    /**
+     * Sort a list of arrays of array(package name, package.xml contents)
+     * by dependency.
+     * @param array
+     */
+    function sortPkgDeps(&$packages)
+    {
+        usort($packages, array('PEAR_Common', '_sortPkgDeps'));
+        $ret = array();
+        foreach($packages as $package) {
+            $ret[] = $package['pkg'];
+        }
+        $packages = $ret;
+    }

+

+    // }}}
+    // {{{ _sortPkgDeps()

+

+    /**
+     * Compare two package's package.xml, and sort
+     * so that dependencies are installed first
+     *
+     * This is a crude compare, real dependency checking is done on install.
+     * The only purpose this serves is to make the command-line
+     * order-independent (you can list a dependent package first, and
+     * installation occurs in the order required)
+     * @access private
+     */
+    function _sortPkgDeps($p1, $p2)
+    {
+        $p1name = $p1['pkg'];
+        $p2name = $p2['pkg'];
+        $p1deps = PEAR_Common::_getPkgDeps($p1);
+        $p2deps = PEAR_Common::_getPkgDeps($p2);
+        if (!count($p1deps) && !count($p2deps)) {
+            return 0; // order makes no difference
+        }
+        if (!count($p1deps)) {
+            return -1; // package 2 has dependencies, package 1 doesn't
+        }
+        if (!count($p2deps)) {
+            return 1; // package 2 has dependencies, package 1 doesn't
+        }
+        // both have dependencies
+        if (in_array($p1name, $p2deps)) {
+            return -1; // put package 1 first
+        }
+        if (in_array($p2name, $p1deps)) {
+            return 1; // put package 2 first
+        }
+        // doesn't really matter if neither depends on the other
+        return 0;
+    }

+

+    // }}}
+    // {{{ _getPkgDeps()

+

+    /**
+     * get an array of package dependency names
+     * @access private
+     */
+    function _getPkgDeps($p)
+    {
+        if (!isset($p['info']['releases'])) {
+            return array();
+        }
+        $rel = array_shift($p['info']['releases']);
+        if (!isset($rel['deps'])) {
+            return array();
+        }
+        $ret = array();
+        foreach($rel['deps'] as $dep) {
+            if ($dep['type'] == 'pkg') {
+                $ret[] = $dep['name'];
+            }
+        }
+        return $ret;
+    }

+

+    // }}}

} ?> Index: pear/PEAR/Installer.php =================================================================== RCS file: /repository/php-src/pear/PEAR/Installer.php,v retrieving revision 1.93 diff -u -r1.93 Installer.php --- pear/PEAR/Installer.php 4 Aug 2003 13:31:35 -0000 1.93 +++ pear/PEAR/Installer.php 4 Aug 2003 22:47:18 -0000 @@ -465,6 +465,33 @@

     // }}}
 +    // {{{ _verifyDeps()
+    +    function _verifyDeps($deps, $options)
+    {
+        // Check dependencies -------------------------------------------
+        if (isset($deps['release_deps']) && empty($options['nodeps'])) {
+            $dep_errors = '';
+            $error = $this->checkDeps($deps, $dep_errors);
+            if ($error == true) {
+                if (empty($options['soft'])) {
+                    $this->log(0, substr($dep_errors, 1));
+                }
+                return $this->raiseError("$pkgname: Dependencies failed");
+            } else if (!empty($dep_errors)) {
+                // Print optional dependencies
+                if (empty($options['soft'])) {
+                    $this->log(0, $dep_errors);
+                }
+            }
+            return true;
+        } else {
+            return false;
+        }
+    }

+

+    // }}}
+         // {{{ install()
      /**

@@ -495,6 +522,7 @@

         }
         $this->registry = &new PEAR_Registry($php_dir);
         $need_download = false;
+        $deps_verified = false;
         //  ==> XXX should be removed later on
         $flag_old_format = false;
         if (preg_match('#^(http|ftp)://#', $pkgfile)) {

@@ -506,6 +534,17 @@

                 {
                     return $this->raiseError("$pkgfile already installed");
                 }
+                $r = new PEAR_Remote($this->config);
+                $info = $r->call('package.info', $pkgfile);
+                // verify dependencies prior to download
+                $release = array_shift($info['releases']);
+                if (isset($release['deps'])) {
+                    $release['release_deps'] = $release['deps'];
+                }
+                $deps_verified = $this->_verifyDeps($release, $options);
+                if (PEAR::isError($deps_verified)) {
+                    return $this->raiseError($deps_verified);
+                }
                 $pkgfile = $this->getPackageDownloadUrl($pkgfile);
                 $need_download = true;
             } else {

@@ -598,21 +637,11 @@

         $pkgname = $pkginfo['package'];
 -        // Check dependencies -------------------------------------------
-        if (isset($pkginfo['release_deps']) && empty($options['nodeps'])) {
-            $dep_errors = '';
-            $error = $this->checkDeps($pkginfo, $dep_errors);
-            if ($error == true) {
-                if (empty($options['soft'])) {
-                    $this->log(0, substr($dep_errors, 1));
-                }
-                return $this->raiseError("$pkgname: Dependencies failed");
-            } else if (!empty($dep_errors)) {
-                // Print optional dependencies
-                if (empty($options['soft'])) {
-                    $this->log(0, $dep_errors);
-                }
-            }
+        if (!$deps_verified) {
+            $deps_verified = $this->_verifyDeps($pkginfo['release_deps'], $options);
+        }
+        if (PEAR::isError($deps_verified)) {
+            return $this->raiseError($deps_verified);
         }
          if (empty($options['force'])) {

Index: pear/PEAR/Command/Install.php =================================================================== RCS file: /repository/php-src/pear/PEAR/Command/Install.php,v retrieving revision 1.46 diff -u -r1.46 Install.php --- pear/PEAR/Command/Install.php 2 Aug 2003 14:59:44 -0000 1.46 +++ pear/PEAR/Command/Install.php 4 Aug 2003 22:47:18 -0000 @@ -242,10 +242,10 @@

         if ($command == 'upgrade') {
             $options[$command] = true;
         }
+        include_once "PEAR/Remote.php";
+        $remote = new PEAR_Remote($this->config);
         if ($command == 'upgrade-all') {
-            include_once "PEAR/Remote.php";
             $options['upgrade'] = true;
-            $remote = new PEAR_Remote($this->config);
             $state = $this->config->get('preferred_state');
             if (empty($state) || $state == 'any') {
                 $latest = $remote->call("package.listLatestReleases");

@@ -273,6 +273,34 @@

                 $this->ui->outputData(array('data' => "Will upgrade $package"), $command);
             }
         }

+

+        // attempt to sort package installation to handle dependencies
+        $depsorted = array(); // packages sorted by dependency
+        $packages = array(); // packages with remote info
+        while (count($params)) {
+            $pkgname = $params[0];
+            if (!preg_match('#^(http|ftp)://#', $pkgname)) {
+                $depsorted[] = $pkgname;
+                unset($params[0]);
+                $params = array_values($params);
+                continue;
+            }
+            $info = $remote->call('package.info', $params[0]);
+            if (!PEAR::isError($info) && count($info['releases'])) {
+                // add to the list of packages to sort
+                $packages[] = array('pkg' => $pkgname, 'info' => $info);
+            } else {
+                // ignore errors, they will be caught below
+                // install this package first, since we don't know
+                // dependency info
+                $depsorted[] = $pkgname;
+            }
+            unset($params[0]); // remove the entry
+            $params = array_values($params); // cycle to the next entry
+        }
+        PEAR_Common::sortPkgDeps($packages);
+        $params = array_merge($depsorted, $packages);

+

         foreach ($params as $pkg) {
             $bn = basename($pkg);
             $info = $this->installer->install($pkg, $options, $this->config);