Skip to content

Shared archive 16 #78

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
reshke merged 4 commits into from
Feb 16, 2026
Merged

Shared archive 16 #78

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1e365e8
Add archive_mode=shared for coordinated WAL archiving
x4m Feb 10, 2026
7eb06cc
Mark ancestor timeline WAL segments as archived
x4m Feb 10, 2026
d249f5c
Optimize ProcessArchivalReport to avoid directory scans
x4m Feb 11, 2026
a6a73b0
Fuse shared archive with ycmdb.shared_archive
x4m Feb 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 29 additions & 7 deletions doc/src/sgml/config.sgml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3687,14 +3687,36 @@ include_dir 'conf.d'
are sent to archive storage by setting
<xref linkend="guc-archive-command"/> or
<xref linkend="guc-archive-library"/>. In addition to <literal>off</literal>,
to disable, there are two modes: <literal>on</literal>, and
<literal>always</literal>. During normal operation, there is no
difference between the two modes, but when set to <literal>always</literal>
the WAL archiver is enabled also during archive recovery or standby
mode. In <literal>always</literal> mode, all files restored from the archive
or streamed with streaming replication will be archived (again). See
<xref linkend="continuous-archiving-in-standby"/> for details.
to disable, there are three modes: <literal>on</literal>, <literal>shared</literal>,
and <literal>always</literal>. During normal operation as a primary, there is no
difference between the three modes, but they differ during archive recovery or
standby mode:
</para>
<itemizedlist>
<listitem>
<para>
<literal>on</literal>: Archives WAL only when running as a primary.
</para>
</listitem>
<listitem>
<para>
<literal>shared</literal>: Coordinates archiving between primary and standby.
The standby defers WAL archival and deletion until the primary confirms
archival via streaming replication. This prevents WAL history loss during
standby promotion in high availability setups. Upon promotion, the standby
automatically starts archiving any remaining unarchived WAL. This mode works
with cascading replication, where each standby coordinates with its immediate
upstream server. See <xref linkend="continuous-archiving-in-standby"/> for details.
</para>
</listitem>
<listitem>
<para>
<literal>always</literal>: Archives all WAL independently, even during recovery.
All files restored from the archive or streamed with streaming physical
replication will be archived (again), regardless of their source.
</para>
</listitem>
</itemizedlist>
<para>
<varname>archive_mode</varname> is a separate setting from
<varname>archive_command</varname> and
Expand Down
72 changes: 49 additions & 23 deletions doc/src/sgml/high-availability.sgml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1377,35 +1377,61 @@ synchronous_standby_names = 'ANY 2 (s1, s2, s3)'
</indexterm>

<para>
When continuous WAL archiving is used in a standby, there are two
different scenarios: the WAL archive can be shared between the primary
and the standby, or the standby can have its own WAL archive. When
the standby has its own WAL archive, set <varname>archive_mode</varname>
When continuous WAL archiving is used in a standby, there are three
different scenarios: the standby can have its own independent WAL archive,
the WAL archive can be shared between the primary and standby, or archiving
can be coordinated between them.
</para>

<para>
For an independent archive, set <varname>archive_mode</varname>
to <literal>always</literal>, and the standby will call the archive
command for every WAL segment it receives, whether it's by restoring
from the archive or by streaming replication. The shared archive can
be handled similarly, but the <varname>archive_command</varname> or <varname>archive_library</varname> must
test if the file being archived exists already, and if the existing file
has identical contents. This requires more care in the
<varname>archive_command</varname> or <varname>archive_library</varname>, as it must
be careful to not overwrite an existing file with different contents,
but return success if the exactly same file is archived twice. And
all that must be done free of race conditions, if two servers attempt
to archive the same file at the same time.
from the archive or by streaming replication.
</para>

<para>
For a shared archive where both primary and standby can write, use
<literal>always</literal> mode as well, but the <varname>archive_command</varname>
or <varname>archive_library</varname> must test if the file being archived
exists already, and if the existing file has identical contents. This requires
more care in the <varname>archive_command</varname> or <varname>archive_library</varname>,
as it must be careful to not overwrite an existing file with different contents,
but return success if the exactly same file is archived twice. And all that must
be done free of race conditions, if two servers attempt to archive the same file
at the same time.
</para>

<para>
For coordinated archiving in high availability setups, use
<varname>archive_mode</varname>=<literal>shared</literal>. In this mode, only
the primary archives WAL segments. The standby creates <literal>.ready</literal>
files for received segments but defers actual archiving. The primary periodically
sends archival status updates to the standby via streaming replication, informing
it which segments have been archived. The standby then marks these as archived
and allows them to be recycled. Upon promotion, the standby automatically starts
archiving any remaining WAL segments that weren't confirmed as archived by the
former primary. This prevents WAL history loss during failover while avoiding
the complexity of coordinating concurrent archiving. This mode works with cascading
replication, where each standby coordinates with its immediate upstream server.
</para>

<para>
If <varname>archive_mode</varname> is set to <literal>on</literal>, the
archiver is not enabled during recovery or standby mode. If the standby
server is promoted, it will start archiving after the promotion, but
will not archive any WAL or timeline history files that
it did not generate itself. To get a complete
series of WAL files in the archive, you must ensure that all WAL is
archived, before it reaches the standby. This is inherently true with
file-based log shipping, as the standby can only restore files that
are found in the archive, but not if streaming replication is enabled.
When a server is not in recovery mode, there is no difference between
<literal>on</literal> and <literal>always</literal> modes.
archiver is not enabled during recovery or standby mode, and this setting
cannot be used on a standby. If a standby with <literal>archive_mode</literal>
set to <literal>on</literal> is promoted, it will start archiving after the
promotion, but will not archive any WAL or timeline history files that it did
not generate itself. To get a complete series of WAL files in the archive, you
must ensure that all WAL is archived before it reaches the standby. This is
inherently true with file-based log shipping, as the standby can only restore
files that are found in the archive, but not if streaming replication is enabled.
</para>

<para>
When a server is not in recovery mode, <literal>on</literal>,
<literal>shared</literal>, and <literal>always</literal> modes all behave
identically, archiving completed WAL segments.
</para>
</sect2>
</sect1>
Expand Down
2 changes: 2 additions & 0 deletions src/backend/access/transam/xlog.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,7 @@ int wal_keep_size_mb = 0;
int XLOGbuffers = -1;
int XLogArchiveTimeout = 0;
int XLogArchiveMode = ARCHIVE_MODE_OFF;
bool ycmdb_shared_archive = false; /* makes archive_mode=on act as shared */
char *XLogArchiveCommand = NULL;
bool EnableHotStandby = false;
bool fullPageWrites = true;
Expand Down Expand Up @@ -196,6 +197,7 @@ const struct config_enum_entry archive_mode_options[] = {
{"always", ARCHIVE_MODE_ALWAYS, false},
{"on", ARCHIVE_MODE_ON, false},
{"off", ARCHIVE_MODE_OFF, false},
{"shared", ARCHIVE_MODE_SHARED, false},
{"true", ARCHIVE_MODE_ON, true},
{"false", ARCHIVE_MODE_OFF, true},
{"yes", ARCHIVE_MODE_ON, true},
Expand Down
17 changes: 13 additions & 4 deletions src/backend/postmaster/pgarch.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,15 @@ pgarch_ArchiverCopyLoop(void)
{
char xlog[MAX_XFN_CHARS + 1];

/*
* In shared archive mode during recovery, the archiver doesn't archive
* files. The primary is responsible for archiving, and the walreceiver
* marks files as .done when the primary confirms archival. After
* promotion, the archiver starts working normally.
*/
if (EffectiveArchiveModeIsShared() && RecoveryInProgress())
return;

/* force directory scan in the first call to pgarch_readyXlog() */
arch_files->arch_files_size = 0;

Expand Down Expand Up @@ -457,10 +466,10 @@ pgarch_ArchiverCopyLoop(void)
continue;
}

if (pgarch_archiveXlog(xlog))
{
/* successful */
pgarch_archiveDone(xlog);
if (pgarch_archiveXlog(xlog))
{
/* successful */
pgarch_archiveDone(xlog);

/*
* Tell the cumulative stats system about the WAL file that we
Expand Down
Loading