Provide a given client with protection against visible database lag. More...

Collaboration diagram for Wikimedia\Rdbms\ChronologyProtector:

Public Member Functions
	__construct ( $cpStash=null, $secret=null, $cliMode=null, $logger=null)

	getClientId ()

	getSessionPrimaryPos (ILoadBalancer $lb)
	Yield client "session consistency" replication position for a new ILoadBalancer.

	getTouched ()
	Whether the request came from a client that recently made database changes (last 10 seconds).

	persistSessionReplicationPositions (&$clientPosIndex=null)
	Persist any staged client "session consistency" replication positions.

	setEnabled ( $enabled)

	setRequestInfo (array $info)

	stageSessionPrimaryPos (ILoadBalancer $lb)
	Update client "session consistency" replication position for an end-of-life ILoadBalancer.

Static Public Member Functions
static	getCPInfoFromCookieValue (?string $value, int $minTimestamp)
	Parse a string conveying the client and write index of the chronology protector data.

static	makeCookieValueFromCPIndex (int $writeIndex, int $time, string $clientId)
	Build a string conveying the client and write index of the chronology protector data.

Public Attributes
const	POSITION_COOKIE_TTL = 10
	Seconds to store position write index cookies (safely less than POSITION_STORE_TTL)

Protected Member Functions
	getStartupSessionPositions ()

	lazyStartup ()
	Load the stored replication positions and touch timestamps for the client.

	mergePositions ( $storedValue, array $shutdownPositions, ?int &$clientPosIndex=null)
	Merge the new replication positions with the currently stored ones (highest wins)

Protected Attributes
string	$clientId
	Hash of client parameters.

string[]	$clientLogInfo
	Map of client information fields for logging.

bool	$enabled = true
	Whether reading/writing session consistency replication positions is enabled.

string	$key
	Storage key name.

LoggerInterface	$logger

array< string, DBPrimaryPos\|null >	$shutdownPositionsByPrimary = []
	Map of (primary server name => position)

array< string, DBPrimaryPos\|null >	$startupPositionsByPrimary = null
	Map of (primary server name => position)

int null	$waitForPosIndex
	Expected minimum index of the last write to the position store.

Detailed Description

Provide a given client with protection against visible database lag.

In a nut shell

This class tries to hide visible effects of database lag. It does this by temporarily remembering the database positions after a client makes a write, and on their next web request we will prefer non-lagged database replicas. When replica connections are established, we wait up to a few seconds for sufficient replication to have occurred, if they were not yet caught up to that same point.

This ensures a consistent ordering of events as seen by a client. Kind of like Hawking's Chronology Protection Agency.

Purpose

For performance and scalability reasons, almost all data is queried from replica databases. Only queries relating to writing data, are sent to a primary database. When rendering a web page with content or activity feeds on it, the very latest information may thus not yet be there. That's okay in general, but if, for example, a client recently changed their preferences or submitted new data, we do our best to make sure their next web response does reflect at least their own recent changes.

How

To explain how it works, we will look at an example lifecycle for a client.

A client is browsing the site. Their web requests are generally read-only and display data from database replicas, which may be a few seconds out of date if a client elsewhere in the world recently modified that same data. If the application is run from multiple data centers, then these web requests may be served from the nearest secondary DC.

A client performs a POST request, perhaps to publish an edit or change their preferences. This request is routed to the primary DC (this is the responsibility of infrastructure outside the web app). There, the data is saved to the primary database, after which the database host will asynchronously replicate this to its replicas in the same and any other DCs.

Toward the end of the response to this POST request, the application takes note of the primary database's current "position", and save this under a "clientId" key in the ChronologyProtector store. The web response will also set two cookies that are similarly short-lived (about ten seconds): UseDC=master and cpPosIndex=<posIndex>@<write time>#<clientId>.

The ten seconds window is meant to account for the time needed for the database writes to have replicated across all active database replicas, including the cross-dc latency for those further away in any secondary DCs. The "clientId" is placed in the cookie to handle the case where the client IP addresses frequently changes between web requests.

Future web requests from the client should fall in one of two categories:

Within the ten second window. Their UseDC cookie will make them return to the primary DC where we access the ChronologyProtector store and use the database "position" to decide which local database replica to use and on-demand wait a split second for replication to catch up if needed.
After the ten second window. They will be routed to the nearest and possibly different DC. Any local ChronologyProtector store existing there will not be interacted with. A random database replica may be used as the client's own writes are expected to have been applied here by now.

Storage requirements

The store used by ChronologyProtector, as configured via $wgMicroStashType, should meet the following requirements:

Low latencies. Nearly all web requests that involve a database connection will unconditionally query this store first. It is expected to respond within the order of one millisecond.
Best effort persistence, without active eviction pressure. Data stored here cannot be obtained elsewhere or recomputed. As such, under normal operating conditions, this store should not be full, and should not evict values before their intended expiry time elapsed.
No replication, local consistency. Each DC may have a fully independent dc-local store associated with ChronologyProtector (no replication across DCs is needed). Local writes must be immediately reflected in subsequent local reads. No intra-dc read lag is allowed.
No redundancy, fast failure. Loss of data will likely be noticeable and disruptive to clients, but the data is not considered essential. Under maintenance or unprecedented load, it is recommended to lose some data, instead of compromising other requirements such as latency or availability for new writes. The fallback is that users may be temporary confused as they observe their own actions as not being immediately reflected. For example, they might change their skin or language preference but still get a one or two page views afterward with the old settings. Or they might have published an edit and briefly not yet see it appear in their contribution history.

Operational requirements

These are the expectations a site administrator must meet for chronology protection:

If the application is run from multiple data centers, then you must designate one of them as the "primary DC". The primary DC is where the primary database is located, from which replication propagates to replica databases in that same DC and any other DCs.
Web requests that use the POST verb, or carry a UseDC=master cookie, must be routed to the primary DC only.

An exception is requests carrying the Promise-Non-Write-API-Action: true header, which use the POST verb for large read queries, but don't actually require the primary DC.

If you have legacy extensions deployed that perform queries on the primary database during GET requests, then you will have to identify a way to route any of its relevant URLs to the primary DC as well, or to accept that their reads do not enjoy chronology protection, and that writes may be slower (due to cross-dc latency). See T91820 for Wikimedia Foundation's routing.

Access: internal

Definition at line 133 of file ChronologyProtector.php.

Constructor & Destructor Documentation

◆ __construct()

Wikimedia\Rdbms\ChronologyProtector::__construct	(	$cpStash = null,
		$secret = null,
		$cliMode = null,
		$logger = null )

Parameters

BagOStuff \| null	$cpStash
string \| null	$secret	Secret string for HMAC hashing [optional]
bool \| null	$cliMode	Whether the context is CLI or not, setting it to true would disable CP
LoggerInterface \| null	$logger

Since: 1.27

Definition at line 194 of file ChronologyProtector.php.

References Wikimedia\Rdbms\ChronologyProtector\$logger.

Member Function Documentation

◆ getClientId()

Wikimedia\Rdbms\ChronologyProtector::getClientId ( )

Returns: string Client ID hash

Since: 1.32

Definition at line 256 of file ChronologyProtector.php.

References Wikimedia\Rdbms\ChronologyProtector\$clientId.

◆ getCPInfoFromCookieValue()

static Wikimedia\Rdbms\ChronologyProtector::getCPInfoFromCookieValue	(	?string	$value,
		int	$minTimestamp )

static

Parse a string conveying the client and write index of the chronology protector data.

Parameters

string \| null	$value	Value of "cpPosIndex" cookie
int	$minTimestamp	Lowest UNIX timestamp that a non-expired value can have

Returns: array (index: int or null, clientId: string or null)

Since: 1.32 in LBFactory, moved to CP in 1.41

Definition at line 583 of file ChronologyProtector.php.

◆ getSessionPrimaryPos()

Wikimedia\Rdbms\ChronologyProtector::getSessionPrimaryPos ( ILoadBalancer $lb )

Yield client "session consistency" replication position for a new ILoadBalancer.

If the stash has a previous primary position recorded, this will try to make sure that the next query to a replica server of that primary will see changes up to that position by delaying execution. The delay may timeout and allow stale data if no non-lagged replica servers are available.

Access: internal: This method should only be called from LBFactory.

Parameters

ILoadBalancer $lb

Returns: DBPrimaryPos|null

Definition at line 282 of file ChronologyProtector.php.

References Wikimedia\Rdbms\ILoadBalancer\getClusterName(), Wikimedia\Rdbms\ILoadBalancer\getServerName(), Wikimedia\Rdbms\ChronologyProtector\getStartupSessionPositions(), and Wikimedia\Rdbms\ServerInfo\WRITER_INDEX.

◆ getStartupSessionPositions()

Wikimedia\Rdbms\ChronologyProtector::getStartupSessionPositions ( )

protected

Returns: array<string,DBPrimaryPos>

Definition at line 434 of file ChronologyProtector.php.

References Wikimedia\Rdbms\ChronologyProtector\$startupPositionsByPrimary, and Wikimedia\Rdbms\ChronologyProtector\lazyStartup().

Referenced by Wikimedia\Rdbms\ChronologyProtector\getSessionPrimaryPos(), and Wikimedia\Rdbms\ChronologyProtector\getTouched().

◆ getTouched()

Wikimedia\Rdbms\ChronologyProtector::getTouched ( )

Whether the request came from a client that recently made database changes (last 10 seconds).

When a user saves an edit or makes other changes to the database, the response to that request contains a short-lived ChronologyProtector cookie (or cpPosIndex query parameter).

If we find such cookie on the current request, and we find any corresponding database positions in the MicroStash, and they are not expired, then we return true.

Since: 1.28 Changed parameter in 1.35. Removed parameter in 1.44.

Returns: bool

Definition at line 415 of file ChronologyProtector.php.

References Wikimedia\Rdbms\ChronologyProtector\getStartupSessionPositions().

◆ lazyStartup()

Wikimedia\Rdbms\ChronologyProtector::lazyStartup ( )

protected

Load the stored replication positions and touch timestamps for the client.

Returns: void

Definition at line 445 of file ChronologyProtector.php.

Referenced by Wikimedia\Rdbms\ChronologyProtector\getStartupSessionPositions().

◆ makeCookieValueFromCPIndex()

static Wikimedia\Rdbms\ChronologyProtector::makeCookieValueFromCPIndex	(	int	$writeIndex,
		int	$time,
		string	$clientId )

static

Build a string conveying the client and write index of the chronology protector data.

Parameters

int	$writeIndex
int	$time	UNIX timestamp; can be used to detect stale cookies (T190082)
string	$clientId	Client ID hash from ILBFactory::shutdown()

Returns: string Value to use for "cpPosIndex" cookie

Since: 1.32 in LBFactory, moved to CP in 1.41

Definition at line 566 of file ChronologyProtector.php.

◆ mergePositions()

Wikimedia\Rdbms\ChronologyProtector::mergePositions	(		$storedValue,
		array	$shutdownPositions,
		?int &	$clientPosIndex = null )

protected

Merge the new replication positions with the currently stored ones (highest wins)

Parameters

	array<string,mixed>\|false	$storedValue Current replication position data
	array<string,DBPrimaryPos>	$shutdownPositions New replication positions
int \| null	&$clientPosIndex	New position write index