Bug #64736 [Opn]: Incorrect calculation in gdColorMatch

Edit report at https://bugs.php.net/bug.php?id=64736&edit=1 ID: 64736 Updated by: cmb@php.net Reported by: sixd@php.net Summary: Incorrect calculation in gdColorMatch Status: Open Type: Bug Package: GD related Operating System: Linux PHP Version: 5.5Git-2013-04-29 (Git) -Assigned To: +Assigned To: pajoye Block user comment: N Private report: N New Comment: It also seems to me, that the calculation is wrong. Assuming that dr, dg, db and da are in range [0, 255], dist would be in the range [0, 510]. Assuming 255^2 is supposed to mean squaring, the argument to sqrt() (in the assignment to dist_perc) is in the range [0, 0.00196], so dist_perc is in the range [0, 0.04428] -- apparently not, what is supposed to represent a percentage. Assuming 255^2 is actually supposed to mean XOR, the respective argument to sqrt() would be in the range [0, 0.6719], so dist_perc is in the range [0, 0.8197] -- looks rather fine, but seems somewhat arbitrarily. With the following, dist_perc would be in the range [0, 1]. const double dist_perc = dist / sqrt(4 * 255 * 255); However, I have some doubts, that the Pythagorean theorem applies to four dimensions in this manner. Anyhow, changing the calculation of dist_perc would constitute a BC break... Previous Comments: ------------------------------------------------------------------------ [2013-04-30 14:48:45] sixd@php.net In C, "^" is the bitwise exclusive OR operator (and has lower precedence than "+"). Pow() is the power function. ------------------------------------------------------------------------ [2013-04-30 05:21:35] pajoye@php.net That's the classic distance calculation in the RGB cube, does ^ not have precedence with the compiler you use? It should (or has in all platforms I use) :). ------------------------------------------------------------------------ [2013-04-29 23:05:06] sixd@php.net Description: ------------ Compilations warnings in gd_crop.c flag a potential calculation issue in gdColorMatch: /home/cjones/php-5.5/ext/gd/libgd/gd_crop.c:340: warning: suggest parentheses around arithmetic in operand of ^ /home/cjones/php-5.5/ext/gd/libgd/gd_crop.c:340: warning: suggest parentheses around arithmetic in operand of ^ The code at line 340 is: const double dist_perc = sqrt(dist / (255^2 + 255^2 + 255^2)); The denominator is currently evaluated as 253. Was it instead intended be (3 * (255 * 255)) == 195075 ? ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=64736&edit=1