Proxy backend that mirrors writes to several internal backends. More...

Inheritance diagram for FileBackendMultiWrite:

Collaboration diagram for FileBackendMultiWrite:

Public Member Functions
	__construct (array $config)
	Construct a proxy backend that consists of several internal backends. More...

	accessibilityCheck (array $paths)
	Check that a set of file paths are usable across all internal backends. More...

	clearCache (array $paths=null)

	concatenate (array $params)

	consistencyCheck (array $paths)
	Check that a set of files are consistent across all internal backends. More...

	directoryExists (array $params)

	fileExists (array $params)

	getDirectoryList (array $params)

	getFeatures ()

	getFileContentsMulti (array $params)

	getFileHttpUrl (array $params)

	getFileList (array $params)

	getFileProps (array $params)

	getFileSha1Base36 (array $params)

	getFileSize (array $params)

	getFileStat (array $params)

	getFileTimestamp (array $params)

	getFileXAttributes (array $params)

	getLocalCopyMulti (array $params)

	getLocalReferenceMulti (array $params)

	getScopedLocksForOps (array $ops, Status $status)

	preloadCache (array $paths)

	preloadFileStat (array $params)

	resyncFiles (array $paths)
	Check that a set of files are consistent across all internal backends and re-synchronize those files against the "multi master" if needed. More...

	streamFile (array $params)

Public Member Functions inherited from FileBackend
	__construct (array $config)
	Create a new backend instance from configuration. More...

	clean (array $params)
	Delete a storage directory if it is empty. More...

	clearCache (array $paths=null)
	Invalidate any in-process file stat and property cache. More...

	concatenate (array $params)
	Concatenate a list of storage files into a single file system file. More...

	copy (array $params, array $opts=array())
	Performs a single copy operation. More...

	create (array $params, array $opts=array())
	Performs a single create operation. More...

	delete (array $params, array $opts=array())
	Performs a single delete operation. More...

	describe (array $params, array $opts=array())
	Performs a single describe operation. More...

	directoryExists (array $params)
	Check if a directory exists at a given storage path. More...

	doOperation (array $op, array $opts=array())
	Same as doOperations() except it takes a single operation. More...

	doOperations (array $ops, array $opts=array())
	This is the main entry point into the backend for write operations. More...

	doQuickOperation (array $op)
	Same as doQuickOperations() except it takes a single operation. More...

	doQuickOperations (array $ops, array $opts=array())
	Perform a set of independent file operations on some files. More...

	fileExists (array $params)
	Check if a file exists at a storage path in the backend. More...

	getContainerStoragePath ($container)
	Get the storage path for the given container for this backend. More...

	getDirectoryList (array $params)
	Get an iterator to list all directories under a storage directory. More...

	getFeatures ()
	Get the a bitfield of extra features supported by the backend medium. More...

	getFileContents (array $params)
	Get the contents of a file at a storage path in the backend. More...

	getFileContentsMulti (array $params)
	Like getFileContents() except it takes an array of storage paths and returns a map of storage paths to strings (or null on failure). More...

	getFileHttpUrl (array $params)
	Return an HTTP URL to a given file that requires no authentication to use. More...

	getFileList (array $params)
	Get an iterator to list all stored files under a storage directory. More...

	getFileProps (array $params)
	Get the properties of the file at a storage path in the backend. More...

	getFileSha1Base36 (array $params)
	Get a SHA-1 hash of the file at a storage path in the backend. More...

	getFileSize (array $params)
	Get the size (bytes) of a file at a storage path in the backend. More...

	getFileStat (array $params)
	Get quick information about a file at a storage path in the backend. More...

	getFileTimestamp (array $params)
	Get the last-modified timestamp of the file at a storage path. More...

	getFileXAttributes (array $params)
	Get metadata about a file at a storage path in the backend. More...

	getJournal ()
	Get the file journal object for this backend. More...

	getLocalCopy (array $params)
	Get a local copy on disk of the file at a storage path in the backend. More...

	getLocalCopyMulti (array $params)
	Like getLocalCopy() except it takes an array of storage paths and returns a map of storage paths to TempFSFile objects (or null on failure). More...

	getLocalReference (array $params)
	Returns a file system file, identical to the file at a storage path. More...

	getLocalReferenceMulti (array $params)
	Like getLocalReference() except it takes an array of storage paths and returns a map of storage paths to FSFile objects (or null on failure). More...

	getName ()
	Get the unique backend name. More...

	getReadOnlyReason ()
	Get an explanatory message if this backend is read-only. More...

	getRootStoragePath ()
	Get the root storage path of this backend. More...

	getScopedFileLocks (array $paths, $type, Status $status, $timeout=0)
	Lock the files at the given storage paths in the backend. More...

	getScopedLocksForOps (array $ops, Status $status)
	Get an array of scoped locks needed for a batch of file operations. More...

	getTopDirectoryList (array $params)
	Same as FileBackend::getDirectoryList() except only lists directories that are immediately under the given directory. More...

	getTopFileList (array $params)
	Same as FileBackend::getFileList() except only lists files that are immediately under the given directory. More...

	getWikiId ()
	Get the wiki identifier used for this backend (possibly empty). More...

	hasFeatures ($bitfield)
	Check if the backend medium supports a field of extra features. More...

	isReadOnly ()
	Check if this backend is read-only. More...

	lockFiles (array $paths, $type, $timeout=0)
	Lock the files at the given storage paths in the backend. More...

	move (array $params, array $opts=array())
	Performs a single move operation. More...

	preloadCache (array $paths)
	Preload persistent file stat cache and property cache into in-process cache. More...

	preloadFileStat (array $params)
	Preload file stat information (concurrently if possible) into in-process cache. More...

	prepare (array $params)
	Prepare a storage directory for usage. More...

	publish (array $params)
	Remove measures to block web access to a storage directory and the container it belongs to. More...

	quickCopy (array $params)
	Performs a single quick copy operation. More...

	quickCreate (array $params)
	Performs a single quick create operation. More...

	quickDelete (array $params)
	Performs a single quick delete operation. More...

	quickDescribe (array $params)
	Performs a single quick describe operation. More...

	quickMove (array $params)
	Performs a single quick move operation. More...

	quickStore (array $params)
	Performs a single quick store operation. More...

	secure (array $params)
	Take measures to block web access to a storage directory and the container it belongs to. More...

	store (array $params, array $opts=array())
	Performs a single store operation. More...

	streamFile (array $params)
	Stream the file at a storage path in the backend. More...

	unlockFiles (array $paths, $type)
	Unlock the files at the given storage paths in the backend. More...

Public Attributes
const	CHECK_SHA1 = 4

const	CHECK_SIZE = 1

const	CHECK_TIME = 2

Public Attributes inherited from FileBackend
const	ATTR_HEADERS = 1
	Bitfield flags for supported features. More...

const	ATTR_METADATA = 2

const	ATTR_UNICODE_PATHS = 4

Protected Member Functions
	doClean (array $params)

	doOperationsInternal (array $ops, array $opts)

	doPrepare (array $params)

	doPublish (array $params)

	doQuickOperationsInternal (array $ops)

	doSecure (array $params)

	fileStoragePathsForOps (array $ops)
	Get a list of file storage paths to read or write for a list of operations. More...

	replicateContainerDirChanges ($path)

	substOpBatchPaths (array $ops, FileBackendStore $backend)
	Substitute the backend name in storage path parameters for a set of operations with that of a given internal backend. More...

	substOpPaths (array $ops, FileBackendStore $backend)
	Same as substOpBatchPaths() but for a single operation. More...

	substPaths ($paths, FileBackendStore $backend)
	Substitute the backend of storage paths with an internal backend's name. More...

	unsubstPaths ($paths)
	Substitute the backend of internal storage paths with the proxy backend's name. More...

Protected Member Functions inherited from FileBackend
	doClean (array $params)

	doOperationsInternal (array $ops, array $opts)

	doPrepare (array $params)

	doPublish (array $params)

	doQuickOperationsInternal (array $ops)

	doSecure (array $params)

	getScopedPHPBehaviorForOps ()
	Enter file operation scope. More...

Protected Attributes
string bool	$autoResync = false

array	$backends = array()
	Prioritized list of FileBackendStore objects. More...

int	$masterIndex = -1
	Index of master backend. More...

array	$noPushDirConts = array()

bool	$noPushQuickOps = false

int	$syncChecks = 0
	Bitfield. More...

Protected Attributes inherited from FileBackend
int	$concurrency
	How many operations can be done in parallel. More...

FileJournal	$fileJournal

LockManager	$lockManager

string	$name
	Unique backend name. More...

string	$parallelize
	When to do operations in parallel. More...

string	$readOnly
	Read-only explanation message. More...

string	$wikiId
	Unique wiki name. More...

Additional Inherited Members
Static Public Member Functions inherited from FileBackend
static	extensionFromPath ($path, $case= 'lowercase')
	Get the final extension from a storage or FS path. More...

static	isPathTraversalFree ($path)
	Check if a relative path has no directory traversals. More...

static	isStoragePath ($path)
	Check if a given path is a "mwstore://" path. More...

static	makeContentDisposition ($type, $filename= '')
	Build a Content-Disposition header value per RFC 6266. More...

static	normalizeStoragePath ($storagePath)
	Normalize a storage path by cleaning up directory separators. More...

static	parentStoragePath ($storagePath)
	Get the parent storage directory of a storage path. More...

static	splitStoragePath ($storagePath)
	Split a storage path into a backend name, a container name, and a relative file path. More...

Static Protected Member Functions inherited from FileBackend
static	normalizeContainerPath ($path)
	Validate and normalize a relative storage path. More...

Detailed Description

Proxy backend that mirrors writes to several internal backends.

This class defines a multi-write backend. Multiple backends can be registered to this proxy backend and it will act as a single backend. Use this when all access to those backends is through this proxy backend. At least one of the backends must be declared the "master" backend.

Only use this class when transitioning from one storage system to another.

Read operations are only done on the 'master' backend for consistency. Write operations are performed on all backends, in the order defined. If an operation fails on one backend it will be rolled back from the others.

Since: 1.19

Definition at line 42 of file FileBackendMultiWrite.php.

Constructor & Destructor Documentation

FileBackendMultiWrite::__construct ( array $config )

Construct a proxy backend that consists of several internal backends.

Locking, journaling, and read-only checks are handled by the proxy backend.

Additional $config params include:

backends : Array of backend config and multi-backend settings. Each value is the config used in the constructor of a FileBackendStore class, but with these additional settings:
- class : The name of the backend class
- isMultiMaster : This must be set for one backend.
- template: : If given a backend name, this will use the config of that backend as a template. Values specified here take precedence.
syncChecks : Integer bitfield of internal backend sync checks to perform. Possible bits include the FileBackendMultiWrite::CHECK_* constants. There are constants for SIZE, TIME, and SHA1. The checks are done before allowing any file operations.
autoResync : Automatically resync the clone backends to the master backend when pre-operation sync checks fail. This should only be used if the master backend is stable and not missing any files. Use "conservative" to limit resyncing to copying newer master backend files over older (or non-existing) clone backend files. Cases that cannot be handled will result in operation abortion.
noPushQuickOps : (hack) Only apply doQuickOperations() to the master backend.
noPushDirConts : (hack) Only apply directory functions to the master backend.

Parameters

array $config

Exceptions

FileBackendError

Definition at line 91 of file FileBackendMultiWrite.php.

References FileBackend\$fileJournal, FileBackend\$name, FileBackend\$wikiId, array(), as, and FileBackendGroup\singleton().

Member Function Documentation

FileBackendMultiWrite::accessibilityCheck ( array $paths )

Check that a set of file paths are usable across all internal backends.

Parameters

array $paths List of storage paths

Returns: Status

Definition at line 276 of file FileBackendMultiWrite.php.

References $path, $status, as, Status\newGood(), and substPaths().

Referenced by doOperationsInternal().

FileBackendMultiWrite::clearCache ( array $paths = null )

Definition at line 656 of file FileBackendMultiWrite.php.

References as, and substPaths().

Referenced by doOperationsInternal().

FileBackendMultiWrite::concatenate ( array $params )

Definition at line 537 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::consistencyCheck ( array $paths )

Check that a set of files are consistent across all internal backends.

Parameters

array $paths List of storage paths

Returns: Status

Definition at line 210 of file FileBackendMultiWrite.php.

References $masterIndex, $params, $path, $status, array(), as, Status\newGood(), substOpPaths(), TS_UNIX, and wfTimestamp().

Referenced by doOperationsInternal().

FileBackendMultiWrite::directoryExists ( array $params )

Definition at line 634 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::doClean ( array $params )

protected

Definition at line 524 of file FileBackendMultiWrite.php.

References $status, as, Status\newGood(), replicateContainerDirChanges(), and substOpPaths().

FileBackendMultiWrite::doOperationsInternal	(	array	$ops,
		array	$opts
	)

finalprotected

Definition at line 143 of file FileBackendMultiWrite.php.

References $masterIndex, $status, accessibilityCheck(), as, clearCache(), consistencyCheck(), FormatJson\encode(), fileStoragePathsForOps(), getScopedLocksForOps(), Status\newGood(), resyncFiles(), substOpBatchPaths(), and wfDebugLog().

FileBackendMultiWrite::doPrepare ( array $params )

protected

Definition at line 485 of file FileBackendMultiWrite.php.

References $status, as, Status\newGood(), replicateContainerDirChanges(), and substOpPaths().

FileBackendMultiWrite::doPublish ( array $params )

protected

Definition at line 511 of file FileBackendMultiWrite.php.

References $status, as, Status\newGood(), replicateContainerDirChanges(), and substOpPaths().

FileBackendMultiWrite::doQuickOperationsInternal ( array $ops )

protected

Definition at line 450 of file FileBackendMultiWrite.php.

References $masterIndex, $status, as, Status\newGood(), and substOpBatchPaths().

FileBackendMultiWrite::doSecure ( array $params )

protected

Definition at line 498 of file FileBackendMultiWrite.php.

References $status, as, Status\newGood(), replicateContainerDirChanges(), and substOpPaths().

FileBackendMultiWrite::fileExists ( array $params )

Definition at line 544 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

Referenced by fileStoragePathsForOps().

FileBackendMultiWrite::fileStoragePathsForOps ( array $ops )

protected

Get a list of file storage paths to read or write for a list of operations.

Parameters

array $ops Same format as doOperations()

Returns: array List of storage paths to files (does not include directories)

Definition at line 362 of file FileBackendMultiWrite.php.

References array(), as, and fileExists().

Referenced by doOperationsInternal().

FileBackendMultiWrite::getDirectoryList ( array $params )

Definition at line 640 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::getFeatures ( )

Definition at line 652 of file FileBackendMultiWrite.php.

References $masterIndex.

FileBackendMultiWrite::getFileContentsMulti ( array $params )

Definition at line 574 of file FileBackendMultiWrite.php.

References $masterIndex, $path, array(), as, substOpPaths(), and unsubstPaths().

FileBackendMultiWrite::getFileHttpUrl ( array $params )

Definition at line 628 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::getFileList ( array $params )

Definition at line 646 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::getFileProps ( array $params )

Definition at line 592 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::getFileSha1Base36 ( array $params )

Definition at line 586 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::getFileSize ( array $params )

Definition at line 556 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::getFileStat ( array $params )

Definition at line 562 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::getFileTimestamp ( array $params )

Definition at line 550 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::getFileXAttributes ( array $params )

Definition at line 568 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::getLocalCopyMulti ( array $params )

Definition at line 616 of file FileBackendMultiWrite.php.

References $masterIndex, $path, array(), as, substOpPaths(), and unsubstPaths().

FileBackendMultiWrite::getLocalReferenceMulti ( array $params )

Definition at line 604 of file FileBackendMultiWrite.php.

References $masterIndex, $path, array(), as, substOpPaths(), and unsubstPaths().

FileBackendMultiWrite::getScopedLocksForOps	(	array	$ops,
		Status	$status
	)

Definition at line 673 of file FileBackendMultiWrite.php.

References $masterIndex, array(), FileBackend\getScopedFileLocks(), LockManager\LOCK_EX, LockManager\LOCK_UW, substOpBatchPaths(), and unsubstPaths().

Referenced by doOperationsInternal().

FileBackendMultiWrite::preloadCache ( array $paths )

Definition at line 663 of file FileBackendMultiWrite.php.

References $masterIndex, and substPaths().

FileBackendMultiWrite::preloadFileStat ( array $params )

Definition at line 668 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::replicateContainerDirChanges ( $path )

protected

Parameters

string $path Storage path

Returns: bool Path container should have dir changes pushed to all backends

Definition at line 479 of file FileBackendMultiWrite.php.

References $path, and list.

Referenced by doClean(), doPrepare(), doPublish(), and doSecure().

FileBackendMultiWrite::resyncFiles ( array $paths )

Check that a set of files are consistent across all internal backends and re-synchronize those files against the "multi master" if needed.

Parameters

array $paths List of storage paths

Returns: Status

Definition at line 301 of file FileBackendMultiWrite.php.

References $masterIndex, $path, $status, array(), as, name, Status\newGood(), substPaths(), and wfDebugLog().

Referenced by doOperationsInternal().

FileBackendMultiWrite::streamFile ( array $params )

Definition at line 598 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

FileBackendMultiWrite::substOpBatchPaths	(	array	$ops,
		FileBackendStore	$backend
	)

protected

Substitute the backend name in storage path parameters for a set of operations with that of a given internal backend.

Parameters

array	$ops	List of file operation arrays
FileBackendStore	$backend

Returns: array

Definition at line 393 of file FileBackendMultiWrite.php.

References array(), as, and substPaths().

Referenced by doOperationsInternal(), doQuickOperationsInternal(), getScopedLocksForOps(), and substOpPaths().

FileBackendMultiWrite::substOpPaths	(	array	$ops,
		FileBackendStore	$backend
	)

protected

Same as substOpBatchPaths() but for a single operation.

Parameters

array	$ops	File operation array
FileBackendStore	$backend

Returns: array

Definition at line 415 of file FileBackendMultiWrite.php.

References array(), and substOpBatchPaths().

Referenced by concatenate(), consistencyCheck(), directoryExists(), doClean(), doPrepare(), doPublish(), doSecure(), fileExists(), getDirectoryList(), getFileContentsMulti(), getFileHttpUrl(), getFileList(), getFileProps(), getFileSha1Base36(), getFileSize(), getFileStat(), getFileTimestamp(), getFileXAttributes(), getLocalCopyMulti(), getLocalReferenceMulti(), preloadFileStat(), and streamFile().

FileBackendMultiWrite::substPaths	(		$paths,
		FileBackendStore	$backend
	)

protected

Substitute the backend of storage paths with an internal backend's name.

Parameters

array \| string	$paths	List of paths or single string path
FileBackendStore	$backend

Returns: array|string

Definition at line 428 of file FileBackendMultiWrite.php.

References name.

Referenced by accessibilityCheck(), clearCache(), preloadCache(), resyncFiles(), and substOpBatchPaths().

FileBackendMultiWrite::unsubstPaths ( $paths )

protected

Substitute the backend of internal storage paths with the proxy backend's name.

Parameters

array | string $paths List of paths or single string path

Returns: array|string

Definition at line 442 of file FileBackendMultiWrite.php.

Referenced by getFileContentsMulti(), getLocalCopyMulti(), getLocalReferenceMulti(), and getScopedLocksForOps().

Member Data Documentation

string bool FileBackendMultiWrite::$autoResync = false

protected

Definition at line 51 of file FileBackendMultiWrite.php.

array FileBackendMultiWrite::$backends = array()

protected

Prioritized list of FileBackendStore objects.

array of (backend index => backends)

Definition at line 45 of file FileBackendMultiWrite.php.

int FileBackendMultiWrite::$masterIndex = -1

protected

Index of master backend.

Definition at line 47 of file FileBackendMultiWrite.php.

Referenced by concatenate(), consistencyCheck(), directoryExists(), doOperationsInternal(), doQuickOperationsInternal(), fileExists(), getDirectoryList(), getFeatures(), getFileContentsMulti(), getFileHttpUrl(), getFileList(), getFileProps(), getFileSha1Base36(), getFileSize(), getFileStat(), getFileTimestamp(), getFileXAttributes(), getLocalCopyMulti(), getLocalReferenceMulti(), getScopedLocksForOps(), preloadCache(), preloadFileStat(), resyncFiles(), and streamFile().

array FileBackendMultiWrite::$noPushDirConts = array()

protected

Definition at line 53 of file FileBackendMultiWrite.php.

bool FileBackendMultiWrite::$noPushQuickOps = false

protected

Definition at line 55 of file FileBackendMultiWrite.php.

int FileBackendMultiWrite::$syncChecks = 0

protected

Bitfield.

Definition at line 49 of file FileBackendMultiWrite.php.

const FileBackendMultiWrite::CHECK_SHA1 = 4

Definition at line 60 of file FileBackendMultiWrite.php.

const FileBackendMultiWrite::CHECK_SIZE = 1

Definition at line 58 of file FileBackendMultiWrite.php.

const FileBackendMultiWrite::CHECK_TIME = 2

Definition at line 59 of file FileBackendMultiWrite.php.

The documentation for this class was generated from the following file:

includes/filebackend/FileBackendMultiWrite.php

Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation