[php-src] PR #23210: ext/pdo_pgsql: Add `Pdo\Pgsql::ATTR_CHUNK_SIZE` for chunked result fetching
| From: | KentarouTakeda | Date: | Tue, 11 Aug 2026 07:08:54 +0000 |
| Subject: | [php-src] PR #23210: ext/pdo_pgsql: Add `Pdo\Pgsql::ATTR_CHUNK_SIZE` for chunked result fetching | ||
| Groups: | php.git-pulls | ||
| Request: | Send a blank email to git-pulls+get-37756@lists.php.net to get a copy of this message | ||
Pull Request: https://github.com/php/php-src/pull/23210
Author: KentarouTakeda
Note: this PR currently also contains the bug fix commits from #23065. The feature is independent of
them, but the existing lazy fetch defects break this feature's tests *too*. Once #23065 is
merged, rebasing will leave only the last commit here.
## Summary
Adds chunked fetching through
Pdo\Pgsql::ATTR_CHUNK_SIZE. It generalizes the lazy fetch
from #15287, and the chunk size lets you choose the balance between memory and speed. ext/pgsql got
this as pg_set_chunked_rows_size in #14571, but there was no way to use it from
ext/pdo_pgsql.
## Benchmark
<details>
<summary>Benchmark code</summary>
```php
<?php
$pdo = new Pdo\Pgsql('pgsql:');
$pdo->exec(<<<SQL
drop table if exists bench;
create temp table bench as select
g as id,
md5(g::text) c1,
'user_'||g as c2,
(g%997)::int as c3,
(g*1.5)::numeric(12,2) as c4,
timestamp '2020-01-01'+(g||' seconds')::interval as c5
from
generate_series(1,500000) g;
analyze bench;
SQL);
$out = fopen('/dev/null', 'w');
$modes = [
'buffered' => [],
'ATTR_PREFETCH => 0' => [PDO::ATTR_PREFETCH => 0],
'ATTR_CHUNK_SIZE => 1' => [Pdo\Pgsql::ATTR_CHUNK_SIZE => 1],
'ATTR_CHUNK_SIZE => 10' => [Pdo\Pgsql::ATTR_CHUNK_SIZE => 10],
'ATTR_CHUNK_SIZE => 100' => [Pdo\Pgsql::ATTR_CHUNK_SIZE => 100],
'ATTR_CHUNK_SIZE => 1000' => [Pdo\Pgsql::ATTR_CHUNK_SIZE => 1000],
];
$held = $rows = $ms = [];
for ($round = -1; $round < 9; $round++) {
foreach ($modes as $name => $options) {
$t0 = hrtime(true);
$statement = $pdo->prepare('select * from bench', $options);
$statement->execute();
$held[$name] = $statement->getAttribute(Pdo\Pgsql::ATTR_RESULT_MEMORY_SIZE);
$rows[$name] = $statement->rowCount();
while ($row = $statement->fetch(PDO::FETCH_ASSOC)) {
fputcsv($out, $row, escape: '');
}
unset($statement);
// round -1 warms up
if ($round >= 0) {
$ms[$name][] = (hrtime(true) - $t0) / 1e6;
}
}
}
fputcsv(STDOUT, ['mode', 'median_ms', 'bytes_held',
'rows_per_fetch'], escape: '');
foreach ($ms as $name => $times) {
sort($times);
fputcsv(STDOUT, [$name, round($times[4]), $held[$name], $rows[$name]], escape: '');
}
```
</details>
Run on a build without --enable-debug (PostgreSQL 18.4 / libpq 18.4):
| mode | median_ms | bytes_held | rows_per_fetch |
| --- | ---: | ---: | ---: |
| buffered | 527 | 97,231,064 | 500,000 |
| ATTR_PREFETCH => 0 | 522 | 3,288 | 1 |
| ATTR_CHUNK_SIZE => 1 | 518 | 3,288 | 1 |
| ATTR_CHUNK_SIZE => 10 | 484 | 3,288 | 10 |
| ATTR_CHUNK_SIZE => 100 | 442 | 19,672 | 100 |
| ATTR_CHUNK_SIZE => 1000 | 429 | 188,632 | 1,000 |
## Usage
Set on the connection:
```php
$pdo->setAttribute(Pdo\Pgsql::ATTR_CHUNK_SIZE, 1000);
```
Set on the statement, overriding the connection level:
```php
$pdo->prepare(
'select * from bench',
[Pdo\Pgsql::ATTR_CHUNK_SIZE => 1000]
);
```
## Design note
The part in bold is where I am not confident that the behavior is right. I would appreciate
feedback.
- ATTR_CHUNK_SIZE overrides ATTR_PREFETCH.
- When ATTR_PREFETCH is left unset and only ATTR_CHUNK_SIZE is given,
this is treated as a request for chunked fetching.
- **It also overrides an explicit ATTR_PREFETCH => 0.** The behavior changes from
lazy to chunk. ATTR_CHUNK_SIZE => 1 behaves the same as lazy, and once a chunk size
is given, I think honoring it is the natural behavior.
- ATTR_CHUNK_SIZE and ATTR_CURSOR => CURSOR_SCROLL are mutually
exclusive.
- A statement-level option takes precedence over the connection level, so a statement-level
CURSOR_SCROLL disables a connection-level ATTR_CHUNK_SIZE.
- When both are given at the statement level, there is no basis for deciding precedence, so a
ValueError is thrown.