Edit report at https://bugs.php.net/bug.php?id=78808&edit=1
ID: 78808
Updated by: cmb@php.net
Reported by: progr-d at yandex dot ru
Summary: [LMDB] MDB_MAP_FULL: Environment mapsize limit
reached
Status: Open
Type: Bug
Package: DBM/DBA related
Operating System: macOS 10.13.6
PHP Version: 7.3.11
Block user comment: N
Private report: N
New Comment:
> contains the fourth argument, but nowhere is there a description> of how to use it
Arguments following the $handler are converted to string and are
passed to the individual DBA drivers, and are driver dependent.
This is indeed not yet documented; let alone which drivers support
which parameters.
> It turns out that this needs to be done here:
Yes. info->argc and info->argv provide the driver specific
arguments which have been passed to dba_open().
Previous Comments:
------------------------------------------------------------------------
[2019-11-13 11:46:03] progr-d at yandex dot ru
Yes, I found a description about this:
> The size should be a multiple of the OS page size. The default is 10485760 bytes. The size of
> the memory map is also the maximum size of the database. The value should be chosen as large as
> possible, to accommodate future growth of the database. This function should be called after
> mdb_env_create() and before mdb_env_open(). It may be called at later times if no transactions are
> active in this process. Note that the library does not check for this condition, the caller must
> ensure it explicitly.
(from http://www.lmdb.tech/doc/group__mdb.html#gaa2506ec8dab3d969b0e609cd82e619e5
)
It turns out that this needs to be done here: https://github.com/php/php-src/blob/PHP-7.2.0/ext/dba/dba_lmdb.c#L61
---------------
Strange thing: 10485760 bytes is 10MB, but my Test script create file only with 1MB size (you can
check it).
---------------
> there likely should be some dba_open() argument
dba_open ( string $path , string $mode [, string $handler [, mixed $... ]] )
contains the fourth argument, but nowhere is there a description of how to use it: https://www.php.net/manual/en/function.dba-open.php
------------------------------------------------------------------------
[2019-11-13 11:30:26] cmb@php.net
From the docs[1]:
| Default size of memory map. This is certainly too small for any
| actual applications. Apps should always set the size explicitly
| using mdb_env_set_mapsize().
So yes, there likely should be some dba_open() argument to allow
for a custom map size.
[1] <http://www.lmdb.tech/doc/group__internal.html#gac08cad5b096925642ca359a6d6f0562a>
------------------------------------------------------------------------
[2019-11-13 10:21:18] progr-d at yandex dot ru
Description:
------------
Presently php not provide any way to change mapsize for lmdb databases and they limited
by size.
How can I fix this? Now, probably, I can only recompile the lmdb library...
PS: build php in macos with lmdb support: https://gist.github.com/saippuakauppias/fac2a33dfe420adb4a6cf19c257eef46
Test script:
---------------
<?php
$lmdb_h = dba_open('test.db', 'c', 'lmdb');
for ($i = 0; $i < 50000; $i++) {
dba_insert('key' . $i, 'value '. $i, $lmdb_h);
}
dba_close($lmdb_h);
Actual result:
--------------
â php prefill_lmdb.php
PHP Warning: dba_insert(key18505,value 18505): MDB_MAP_FULL: Environment mapsize limit reached in
/Users/usr/prefill_lmdb.php on line 6
Warning: dba_insert(key18505,value 18505): MDB_MAP_FULL: Environment mapsize limit reached in
/Users/usr/prefill_lmdb.php on line 6
[1] 42370 segmentation fault php prefill_lmdb.php
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=78808&edit=1