Re: Patch: automatic dependency resolution, and new install options --alldeps
| From: | Greg Beaver | Date: | Sat, 23 Aug 2003 05:16:41 +0000 |
| Subject: | Re: Patch: automatic dependency resolution, and new install options --alldeps | ||
| References: | 1 | Groups: | php.pear.dev |
| Request: | Send a blank email to pear-dev+get-20424@lists.php.net to get a copy of this message | ||
One other change I forgot to mention: the default behavior of install has changed because of the nature of the patch. Install is now an alias for upgrade and vice-versa (you can use the syntax pear install SOAP to upgrade it, or pear install -a SOAP to upgrade SOAP and all its dependencies). This is a big change that requires serious thought and testing.
Regards,
Greg
Greg Beaver wrote:
Hi, As I mentioned a few days ago in another thread, I've been struggling with how many dependencies phpDocumentor 2.0 can have on existing PEAR packages, and how to solve this problem. Well, the solution to one major stumbling block is attached to this email :). The attached patch brings PEAR installation to the same level of sophistication that apt possesses. For the first time, automatic dependency resolution and install is possible. It is experimental, but fully tested. I'd love it if a few brave souls would be willing to back up their existing pear/PEAR directory and try this patch out, to assist in the development. I'm also not particularly happy with the length of the command-line to PEAR_Installer::download() and if anyone can see a refinement, I've love to get that in here. Note that the patch was made against the HEAD of php-src/pear, *NOT* version 1.2.1. What does the patch do? ----------------------- * makes the order of packages passed to pear install/pear upgrade unimportant In previous versions this would fail: pear install HTTP_Request Net_URL With this patch, it correctly switches the install order so that dependencies are installed first. * adds the new install options --alldeps and --allreqdeps --alldeps instructs pear to recursively find and install all required and all optional dependencies --allreqdeps instructs pear to recursively find and install all required dependencies only usage example: pear install --alldeps SOAP usage example: pear install --a SOAP These examples will find the dependencies of SOAP: Mail_Mime, HTTP_Request, Net_URL, and Net_DIME and download them. Then it will find all the dependencies of the dependencies, and download Net_Socket as a result (Net_URL will not be downloaded twice). If any of these dependencies are installed and up to date, they will not be downloaded. If they are installed and not up to date, they will be upgraded. Finally, the packages will be sorted into the order needed to install them, and installed. I've tested with both Cache (3 levels of dependency) and SOAP (2 levels plus common dependency across levels) usage example: pear install --allreqdeps pearcvs/pear/Perm_LiveUser/package.xml usage example: pear install --A pearcvs/pear/Perm_LiveUser/package.xml These examples assume pearcvs/ is a cvs checkout of the pear/ module from cvs.php.net This one will install LiveUser 0.8 from CVS, and skip the download and install of MDB, DB and HTTP_Request * splits PEAR_Installer::install() into 3 methods: private _downloadFile(), public download() and public install(). This split allows the download of all files prior to any installation attempt, to allow full dependency resolution and sorting for installation. * Numerous important failure tests - failure on package not existing, or having no releases yet - failure if the preferred state is too high for dependencies to be installed (Net_DIME only has beta releases, for example, so if preferred_state is stable, it will fail on installation) What doesn't the patch do? -------------------------- * Major issue: no check for major version upgrade/install - waiting for resolution on how major upgrade/install will be handled (this is an existing issue in the pear installer, so I decided not to "fix" it) * No change is made to pear uninstall. "pear uninstall Net_URL HTTP_Request" will still fail due to incorrect ordering of packages on the command-line, and "pear uninstall SOAP" will not attempt to uninstall any dependencies. I didn't feel comfortable with the safety of adding --alldeps or --allreqdeps to uninstall, but if there is a demand for it, it can be easily added. (uninstalling dependencies is MUCH simpler than installing them). :) Greg ------------------------------------------------------------------------ Index: pear/PEAR/Common.php =================================================================== RCS file: /repository/php-src/pear/PEAR/Common.php,v retrieving revision 1.100 diff -u -r1.100 Common.php --- pear/PEAR/Common.php 13 Aug 2003 21:12:15 -0000 1.100 +++ pear/PEAR/Common.php 23 Aug 2003 04:45:11 -0000 @@ -1358,6 +1358,30 @@} // }}} + // {{{ betterStates()++ /** + * Return an array containing all of the states that are more stable than + * or equal to the passed in state + * + * @param string Release state + * @param boolean Determines whether to include $state in the list + * @return false|array False if $state is not a valid release state + */ + function betterStates($state, $include = false) + { + static $states = array('devel', 'snapshot', 'alpha', 'beta', 'stable'); + $i = array_search($state, $states); + if ($i === false) { + return false; + } + if ($include) { + $i--; + } + return array_slice($states, $i + 1); + }++ // }}} // {{{ detectDependencies() function detectDependencies($any, $status_callback = null)@@ -1688,6 +1712,104 @@call_user_func($callback, 'done', $bytes); } return $dest_file; + }++ // }}} + // {{{ sortPkgDeps()++ /** + * Sort a list of arrays of array(downloaded packagefilename) by dependency. + * + * It also removes duplicate dependencies + * @param array + * @return array array of array(packagefilename, package.xml contents) + */ + function sortPkgDeps(&$packages) + { + foreach($packages as $packagefile) { + if (!is_array($packagefile)) { + $ret[] = array('file' => $packagefile, + 'info' => $a = $this->infoFromAny($packagefile), + 'pkg' => $a['package']); + } else { + $ret[] = $packagefile; + } + } + $checkdupes = array(); + $newret = array(); + foreach($ret as $i => $p) { + if (!isset($checkdupes[$p['info']['package']])) { + $checkdupes[$p['info']['package']][] = $i; + $newret[] = $p; + } + } + + usort($newret, array('PEAR_Common', '_sortPkgDeps')); + $packages = $newret; + }++ // }}} + // {{{ _sortPkgDeps()++ /** + * Compare two package's package.xml, and sort + * so that dependencies are installed first + * + * This is a crude compare, real dependency checking is done on install. + * The only purpose this serves is to make the command-line + * order-independent (you can list a dependent package first, and + * installation occurs in the order required) + * @access private + */ + function _sortPkgDeps($p1, $p2) + { + $p1name = $p1['info']['package']; + $p2name = $p2['info']['package']; + $p1deps = PEAR_Common::_getPkgDeps($p1); + $p2deps = PEAR_Common::_getPkgDeps($p2); + if (!count($p1deps) && !count($p2deps)) { + return 0; // order makes no difference + } + if (!count($p1deps)) { + return -1; // package 2 has dependencies, package 1 doesn't + } + if (!count($p2deps)) { + return 1; // package 2 has dependencies, package 1 doesn't + } + // both have dependencies + if (in_array($p1name, $p2deps)) { + return -1; // put package 1 first + } + if (in_array($p2name, $p1deps)) { + return 1; // put package 2 first + } + // doesn't really matter if neither depends on the other + return 0; + }++ // }}} + // {{{ _getPkgDeps()++ /** + * get an array of package dependency names + * @access private + */ + function _getPkgDeps($p) + { + if (!isset($p['info']['releases'])) { + return array(); + } + $rel = array_shift($p['info']['releases']); + if (!isset($rel['deps'])) { + return array(); + } + $ret = array(); + foreach($rel['deps'] as $dep) { + if ($dep['type'] == 'pkg') { + $ret[] = $dep['name']; + } + } + return $ret; } // }}}Index: pear/PEAR/Installer.php =================================================================== RCS file: /repository/php-src/pear/PEAR/Installer.php,v retrieving revision 1.98 diff -u -r1.98 Installer.php --- pear/PEAR/Installer.php 17 Aug 2003 18:12:43 -0000 1.98 +++ pear/PEAR/Installer.php 23 Aug 2003 04:45:12 -0000 @@ -464,26 +464,10 @@} // }}}-- // {{{ install()-- /** - * Installs the files within the package file specified. - * - * @param $pkgfile path to the package file - * - * @return array package info if successful, null if not - */-- function install($pkgfile, $options = array()) + // {{{ _downloadFile() + function _downloadFile($pkgfile, &$config, $options, &$errors) { - // recognized options: - // - force : force installation - // - register-only : update registry but don't install files - // - upgrade : upgrade existing install - // - soft : fail silently - // - $php_dir = $this->config->get('php_dir'); + $php_dir = $config->get('php_dir'); if (isset($options['installroot'])) { if (substr($options['installroot'], -1) == DIRECTORY_SEPARATOR) { $options['installroot'] = substr($options['installroot'], 0, -1);@@ -495,8 +479,6 @@} $this->registry = &new PEAR_Registry($php_dir); $need_download = false; - // ==> XXX should be removed later on - $flag_old_format = false; if (preg_match('#^(http|ftp)://#', $pkgfile)) { $need_download = true; } elseif (!@is_file($pkgfile)) {@@ -504,22 +486,24 @@if ($this->registry->packageExists($pkgfile) && empty($options['upgrade']) && empty($options['force'])) { - return $this->raiseError("$pkgfile already installed"); + $errors[] = "$pkgfile already installed"; + return; } $pkgfile = $this->getPackageDownloadUrl($pkgfile); $need_download = true; } else { if (strlen($pkgfile)) { - return $this->raiseError("Could not open the package file: $pkgfile"); + $errors[] = "Could not open the package file: $pkgfile"; } else { - return $this->raiseError("No package file given"); + $errors[] = "No package file given"; } + return; } } // Download package ----------------------------------------------- if ($need_download) { - $downloaddir = $this->config->get('download_dir'); + $downloaddir = $config->get('download_dir'); if (empty($downloaddir)) { if (PEAR::isError($downloaddir = System::mktemp('-d'))) { return $downloaddir;@@ -533,6 +517,209 @@} $pkgfile = $file; } + return $pkgfile; + }++ // }}} + // {{{ download()++ /** + * Download any files and their dependencies, if necessary + * + * @param array a mixed list of package names, local files, or package.xml + * @param PEAR_Config + * @param array options from the command line + * @param array this is the array that will be populated with packages to + * install. Format of each entry: + * + * <code> + * array('pkg' => 'package_name', 'file' => '/path/to/local/file', + * 'info' => array() // parsed package.xml + * ); + * </code> + * @param array this will be populated with any error messages + * @param string command name (upgrade/install/upgrade-all) + * @param false private recursion variable + * @param false private recursion variable + * @param false private recursion variable + */ + function download($packages, &$options, &$config, &$installpackages, + &$errors, $command, $installed = false, $willinstall = false, $state = false) + { + // recognized options: + // - allreqdeps : install all required dependencies as well + // - alldeps : install all dependencies, including optional + // + if (!$willinstall) { + $willinstall = array(); + } + $mywillinstall = array();++ // download files in this list if necessary + foreach($packages as $pkgfile) { + $pkgfile = $this->_downloadFile($pkgfile, $config, $options, $errors); + if (PEAR::isError($pkgfile)) { + return $pkgfile; + } + $tempinfo = $this->infoFromAny($pkgfile); + if (isset($options['alldeps']) || isset($options['allreqdeps'])) { + // ignore dependencies if there are any errors + if (!PEAR::isError($tempinfo)) { + $mywillinstall[strtolower($tempinfo['package'])] = @$tempinfo['release_deps']; + } + } + $installpackages[] = array('pkg' => $tempinfo['package'], + 'file' => $pkgfile, 'info' => $tempinfo); + }++ // extract dependencies from downloaded files and then download them + // if necessary + if (isset($options['alldeps']) || isset($options['allreqdeps'])) { + $reg = new PEAR_Registry($config->get('php_dir')); + if (!$installed) { + $state = $config->get('preferred_state'); + $installed = $reg->listPackages(); + array_walk($installed, create_function('&$v,$k','$v = strtolower($v);')); + $installed = array_flip($installed); + } + include_once "PEAR/Remote.php"; + $remote = new PEAR_Remote($config); + $deppackages = array(); + // construct the list of dependencies for each file + foreach ($mywillinstall as $package => $alldeps) { + if (!is_array($alldeps)) { + continue; + } + foreach($alldeps as $info) { + if ($info['type'] != 'pkg') { + continue; + } + if (!isset($options['alldeps']) && isset($info['optional']) && + $info['optional'] == 'yes') { + // skip optional deps + $this->log(0, "skipping Package $package optional dependency $info[name]"); + continue; + } + // get releases + $releases = $remote->call('package.info', $info['name'], 'releases'); + if (PEAR::isError($releases)) { + return $releases; + } + if (!count($releases)) { + if (!isset($installed[strtolower($info['name'])])) { + $errors[] = "Package $package dependency $info[name] ". + "has no releases"; + } + continue; + } + $found = false; + $save = $releases; + while(count($releases) && !$found) { + if (!empty($state) && $state != 'any') { + list($release_version,$release) = each($releases); + if ($state != $release['state'] && + !in_array($release['state'], + $this->betterStates($state))) { + // drop this release - it ain't stable enough + array_shift($releases); + } else { + $found = true; + } + } else { + $found = true; + } + } + if (!count($releases) && !$found) { + $get = array(); + foreach($save as $release) { + $get = array_merge($get, + $this->betterStates($release['state'], true)); + } + $savestate = array_shift($get); + $errors[] = "Release for $package dependency $info[name] " . + "has state '$savestate', requires $state"; + continue; + } + if (in_array(strtolower($info['name']), $willinstall) || + isset($mywillinstall[strtolower($info['name'])])) { + // skip upgrade check for packages we will install + continue; + } + if (!isset($installed[strtolower($info['name'])])) { + // skip upgrade check for packages we don't have installed + $deppackages[] = $info['name']; + continue; + }++ // see if a dependency must be upgraded + $inst_version = $reg->packageInfo($info['name'], 'version'); + if (!isset($info['version'])) { + // this is a rel='has' dependency, check against latest + if (version_compare($release_version, $inst_version, 'le')) { + continue; + } else { + $deppackages[] = $info['name']; + continue; + } + } + if (version_compare($info['version'], $inst_version, 'le')) { + // installed version is up-to-date + continue; + } + $deppackages[] = $info['name']; + } // foreach($alldeps + } // foreach($willinstall++ if (count($deppackages)) { + // check dependencies' dependencies + // combine the list of packages to install + $temppack = array(); + foreach($installpackages as $p) { + $temppack[] = strtolower($p['info']['package']); + } + foreach($deppackages as $pack) { + $temppack[] = strtolower($pack); + } + $willinstall = array_merge($willinstall, $temppack); + $this->download($deppackages, $options, $config, $installpackages, + $errors, $command, $installed, $willinstall, $state); + } + } // if --alldeps or --allreqdeps + }++ // }}} + // {{{ install()++ /** + * Installs the files within the package file specified. + * + * @param $pkgfile path to the package file + * + * @return array package info if successful, null if not + */++ function install($pkgfile, $options = array()) + { + // recognized options: + // - force : force installation + // - register-only : update registry but don't install files + // - upgrade : upgrade existing install + // - soft : fail silently + // + $php_dir = $this->config->get('php_dir'); + if (isset($options['installroot'])) { + if (substr($options['installroot'], -1) == DIRECTORY_SEPARATOR) { + $options['installroot'] = substr($options['installroot'], 0, -1); + } + $php_dir = $this->_prependPath($php_dir, $options['installroot']); + $this->installroot = $options['installroot']; + } else { + $this->installroot = ''; + } + $this->registry = &new PEAR_Registry($php_dir); + $need_download = false; + // ==> XXX should be removed later on + $flag_old_format = false; if (substr($pkgfile, -4) == '.xml') { $descfile = $pkgfile;@@ -644,19 +831,21 @@} } else { // checks to do only when upgrading packages - if (!$this->registry->packageExists($pkgname)) { +/* if (!$this->registry->packageExists($pkgname)) { return $this->raiseError("$pkgname not installed"); - } - $v1 = $this->registry->packageInfo($pkgname, 'version'); - $v2 = $pkginfo['version']; - $cmp = version_compare("$v1", "$v2", 'gt'); - if (empty($options['force']) && !version_compare("$v2", "$v1", 'gt')) { - return $this->raiseError("upgrade to a newer version ($v2 is not newer than $v1)"); - } - if (empty($options['register-only'])) { - // when upgrading, remove old release's files first: - if (PEAR::isError($err = $this->_deletePackageFiles($pkgname))) { - return $this->raiseError($err); + }*/ + if ($this->registry->packageExists($pkgname)) { + $v1 = $this->registry->packageInfo($pkgname, 'version'); + $v2 = $pkginfo['version']; + $cmp = version_compare("$v1", "$v2", 'gt'); + if (empty($options['force']) && !version_compare("$v2", "$v1", 'gt')) { + return $this->raiseError("upgrade to a newer version ($v2 is not newer than $v1)"); + } + if (empty($options['register-only'])) { + // when upgrading, remove old release's files first: + if (PEAR::isError($err = $this->_deletePackageFiles($pkgname))) { + return $this->raiseError($err); + } } } }@@ -743,7 +932,7 @@} // Register that the package is installed ----------------------- - if (empty($options['upgrade'])) { +/* if (empty($options['upgrade'])) { // if 'force' is used, replace the info in registry if (!empty($options['force']) && $this->registry->packageExists($pkgname)) { $this->registry->deletePackage($pkgname);@@ -751,7 +940,18 @@$ret = $this->registry->addPackage($pkgname, $pkginfo); } else { $ret = $this->registry->updatePackage($pkgname, $pkginfo, false); + }*/ + // if 'force' is used, replace the info in registry + $pkg_installed = $this->registry->packageExists($pkgname); + if (!empty($options['force']) && $pkg_installed) { + $this->registry->deletePackage($pkgname); } + if (!$pkg_installed) { + $ret = $this->registry->addPackage($pkgname, $pkginfo); + } else { + $ret = $this->registry->updatePackage($pkgname, $pkginfo, false); + }+if (!$ret) { return null; }Index: pear/PEAR/Command/Install.php =================================================================== RCS file: /repository/php-src/pear/PEAR/Command/Install.php,v retrieving revision 1.47 diff -u -r1.47 Install.php --- pear/PEAR/Command/Install.php 5 Aug 2003 15:36:07 -0000 1.47 +++ pear/PEAR/Command/Install.php 23 Aug 2003 04:45:13 -0000 @@ -69,6 +69,14 @@'ignore-errors' => array( 'doc' => 'force install even if there were errors', ), + 'alldeps' => array( + 'shortopt' => 'a', + 'doc' => 'install all required and optional dependencies', + ), + 'allreqdeps' => array( + 'shortopt' => 'A', + 'doc' => 'install all required dependencies', + ), ), 'doc' => '<package> ...Installs one or more PEAR packages. You can specify a package to @@ -273,9 +281,23 @@$this->ui->outputData(array('data' => "Will upgrade $package"), $command); } } - foreach ($params as $pkg) { - $bn = basename($pkg); - $info = $this->installer->install($pkg, $options, $this->config); + $errors = array(); + $downloaded = array(); + if (isset($options['alldeps']) || isset($options['allreqdeps'])) { + $options['upgrade'] = true; + } + $this->installer->download($params, $options, $this->config, $downloaded, + $errors, $command); + if (count($errors)) { + $err['data'] = array($errors); + $err['headline'] = 'Install Errors'; + $this->ui->outputData($err); + return $this->raiseError("$command failed"); + } + $this->installer->sortPkgDeps($downloaded); + foreach ($downloaded as $pkg) { + $bn = basename($pkg['file']); + $info = $this->installer->install($pkg['file'], $options, $this->config); if (is_array($info)) { if ($this->config->get('verbose') > 0) { $label = "$info[package] $info[version]";