#17490 [Com]: serialize() generates a string that unserialize() can't handle

ID: 17490 Comment by: dougqh@incogen.com Reported By: weyrick@roadsend.com Status: Feedback Bug Type: Strings related Operating System: linux rh7.2 PHP Version: 4.2.1 New Comment: I tried the latest CVS snap shot as requested to no avail. I actually reworked the serializer to fix the problem. I have not submitted my changes because of some odd behavior, compiler warnings, and other things I wanted to clean up first. I have a question about the array serialization, which appears to be the culprit. I posted this to the PHP-dev list a couple times and received no replies. My email to the list is provided below. Any insights would be appreciated. The cause of the problem appears to be that zend_hash_num_elements was returning a number smaller than the actual number of elements being iterated across using the zend_hash_*_ex functions. This caused the serializer to store a number of elements smaller than the actual number of elements written. This in turn leads to the unserializer failing later. I would like to know if there is some reason why zend_hash_num_elements would be expected to return a value smaller than the number of actual elements iterated across using the zend_hash_*_ex functions. This problem is rare, but was happening sporadically. Below is the section of code that is displaying this behavior. If you see any obvious errors please let me know. static inline void php_var_serialize_hash_all( smart_str *buf, zval **struc, HashTable *var_hash TSRMLS_DC ) { int numElements; int cur; int actualCount; smart_str tmp = {0}; numElements = zend_hash_num_elements( HASH_OF( *struc ) ); /* There is a problem with the value returned by zend_hash_num_elements not returning a value equivalent to the number elements actually iterated across using zend_hash_*_ex functions */ /* Solution iterate filling a temporary buffer and remembering the count - then copy the temporary buffer into the actual buffer */ if ( numElements > 0 ) { char *key; zval **data; ulong index; uint key_len; HashPosition pos; for ( /* initialize */ zend_hash_internal_pointer_reset_ex( HASH_OF( *struc ), &pos ), actualCount = 0; /* test */; /* advance */ zend_hash_move_forward_ex( HASH_OF( *struc ), &pos ), ++actualCount ) { cur = zend_hash_get_current_key_ex( HASH_OF( *struc ), &key, &key_len, &index, 0, &pos ); if ( cur == HASH_KEY_NON_EXISTANT ) break; switch ( cur ) { case HASH_KEY_IS_LONG: php_var_serialize_long( &tmp, index ); break; case HASH_KEY_IS_STRING: php_var_serialize_string( &tmp, key, key_len - 1 ); break; } /* we should still add element even if it's not OK, since we already wrote the length of the array before */ if ( ( zend_hash_get_current_data_ex( HASH_OF( *struc ), ( void ** ) &data, &pos ) != SUCCESS ) || ( ! data ) || ( data == struc ) ) { php_var_serialize_null( &tmp ); } else { php_var_serialize_intern( &tmp, data, var_hash TSRMLS_CC ); } } } else { actualCount = 0; } smart_str_append_long( buf, actualCount ); smart_str_appendc( buf, TYPE_VALUE_DELIMITER_CHAR ); smart_str_appendc( buf, OPEN_HASH_CHAR ); smart_str_appendl( buf, tmp.c, tmp.len ); smart_str_appendc( buf, CLOSE_HASH_CHAR ); /* clean up tmp */ smart_str_free_ex( &tmp, 1 ); } Previous Comments: ------------------------------------------------------------------------ [2002-10-10 22:54:09] iliaa@php.net Please try using this CVS snapshot: http://snaps.php.net/php4-latest.tar.gz For Windows: http://snaps.php.net/win32/php4-win32-latest.zip This bug may be related to locale issues that were addressed in the latest CVS. ------------------------------------------------------------------------ [2002-09-25 06:34:03] sas@php.net Not session related, reclassified. ------------------------------------------------------------------------ [2002-09-16 09:28:55] dougqh@incogen.com My problem is not improperly using __sleep. Referencing complexity definitely seems to be part of the problem. I am serializing multiple groups of objects for different purposes. Some of my object structures have relatively simple referencing complexity, and I can get away with using the standard serializer on those objects. Others are far more free form and complicated, and I cannot safely use the standard serializer on those objects. At one point, I created a failure in my "simple" objects (the ones using the standard serializer) when I tried to add an associative array of numeric arrays of objects to the top-level object being serialized. The objects in the numeric array pointed back to the top-level object containing the associative array. Old referencing structure ... object A -member is-> associative array -elements are-> arrays -elements are-> objects -points to-> object A I resolved this problem by removing the associative array from the equation and creating dynamically named variables to directly contain the object arrays. New referencing structure ... object A -members are-> arrays -elements are-> objects -points to-> object A ------------------------------------------------------------------------ [2002-09-09 13:09:57] weyrick@roadsend.com i'm not sure about the others who were having similiar problems, but the problem i was having has been solved. it turned out to be a misuse of the __sleep() method in my base object. i mistakenly thought the method should have been unsetting variables not to be serialized, when in fact it needed to return an array of variables TO serialize. once this mistake was caught and fixed, serialization of my objects started working. it would still seem to be a bug in PHP, however, to have the actual unserialize call fail directly after a serialize call. this should give a good starting point for the developers to look at, and others who were having a similiar problem can check their code. ------------------------------------------------------------------------ [2002-08-22 14:17:58] dougqh@incogen.com As others have said, the problem seems linked to the complexity of the variable being serialized and not strictly the size of the result string. Of note, the WDDX serializer serialized and unserialized my object array without any problems, but its lack of reference support may account for that. I actually needed references preserved, so for now I am using a combination of pre-serialization and post-unserialization processing in combination with WDDX as a work around. ------------------------------------------------------------------------ The remainder of the comments for this report are too long. To view the rest of the comments, please view the bug report online at http://bugs.php.net/17490 -- Edit this bug report at http://bugs.php.net/?id=17490&edit=1