Storage Engine API
Classes | Public Types | Public Member Functions | List of all members
mongo::StorageEngine Class Referenceabstract

The StorageEngine class is the top level interface for creating a new storage engine. More...

#include <storage_engine.h>

Inheritance diagram for mongo::StorageEngine:
mongo::KVStorageEngine mongo::MMAPV1Engine

Detailed Description

The StorageEngine class is the top level interface for creating a new storage engine.

All StorageEngine(s) must be registered by calling registerFactory in order to possibly be activated.

Classes

class  Factory
 The interface for creating new instances of storage engines. More...
 

Public Types

typedef std::pair< std::string, std::string > CollectionIndexNamePair
 

Public Member Functions

virtual ~StorageEngine ()
 The destructor should only be called if we are tearing down but not exiting the process. More...
 
virtual void finishInit ()
 Called after the globalStorageEngine pointer has been set up, before any other methods are called. More...
 
virtual RecoveryUnit * newRecoveryUnit ()=0
 Returns a new interface to the storage engine's recovery unit. More...
 
virtual void listDatabases (std::vector< std::string > *out) const =0
 List the databases stored in this storage engine. More...
 
virtual DatabaseCatalogEntry * getDatabaseCatalogEntry (OperationContext *opCtx, StringData db)=0
 Return the DatabaseCatalogEntry that describes the database indicated by 'db'. More...
 
virtual bool supportsDocLocking () const =0
 Returns whether the storage engine supports its own locking locking below the collection level. More...
 
virtual bool supportsDBLocking () const
 Returns whether the storage engine supports locking at a database level. More...
 
virtual bool supportsCappedCollections () const
 Returns whether the storage engine supports capped collections. More...
 
virtual bool isDurable () const =0
 Returns whether the engine supports a journalling concept or not. More...
 
virtual bool isEphemeral () const =0
 Returns true if the engine does not persist data to disk; false otherwise. More...
 
virtual bool isMmapV1 () const
 Only MMAPv1 should override this and return true to trigger MMAPv1-specific behavior. More...
 
virtual void loadCatalog (OperationContext *opCtx)
 Populates and tears down in-memory data structures, respectively. More...
 
virtual void closeCatalog (OperationContext *opCtx)
 
virtual Status closeDatabase (OperationContext *opCtx, StringData db)=0
 Closes all file handles associated with a database. More...
 
virtual Status dropDatabase (OperationContext *opCtx, StringData db)=0
 Deletes all data and metadata for a database. More...
 
virtual int flushAllFiles (OperationContext *opCtx, bool sync)=0
 
virtual Status beginBackup (OperationContext *opCtx)
 Transitions the storage engine into backup mode. More...
 
virtual void endBackup (OperationContext *opCtx)
 Transitions the storage engine out of backup mode. More...
 
virtual Status repairRecordStore (OperationContext *opCtx, const std::string &ns)=0
 Recover as much data as possible from a potentially corrupt RecordStore. More...
 
virtual void cleanShutdown ()=0
 This method will be called before there is a clean shutdown. More...
 
virtual SnapshotManager * getSnapshotManager () const
 Returns the SnapshotManager for this StorageEngine or NULL if not supported. More...
 
virtual void setJournalListener (JournalListener *jl)=0
 Sets a new JournalListener, which is used by the storage engine to alert the rest of the system about journaled write progress. More...
 
virtual bool supportsRecoverToStableTimestamp () const
 Returns whether the storage engine supports "recover to stable timestamp". More...
 
virtual bool supportsReadConcernSnapshot () const
 Returns true if the storage engine supports the readConcern level "snapshot". More...
 
virtual StatusWith< Timestamp > recoverToStableTimestamp (OperationContext *opCtx)
 Recovers the storage engine state to the last stable timestamp. More...
 
virtual boost::optional< Timestamp > getRecoveryTimestamp () const
 Returns the stable timestamp that the storage engine recovered to on startup. More...
 
virtual boost::optional< Timestamp > getLastStableCheckpointTimestamp () const
 Returns a timestamp that is guaranteed to be persisted to disk in a checkpoint. More...
 
virtual void setStableTimestamp (Timestamp timestamp)
 Sets the highest timestamp at which the storage engine is allowed to take a checkpoint. More...
 
virtual void setInitialDataTimestamp (Timestamp timestamp)
 Tells the storage engine the timestamp of the data at startup. More...
 
virtual void setOldestTimestampFromStable ()
 Uses the current stable timestamp to set the oldest timestamp for which the storage engine must maintain snapshot history through. More...
 
virtual void setOldestTimestamp (Timestamp timestamp)
 Sets the oldest timestamp for which the storage engine must maintain snapshot history through. More...
 
virtual bool isCacheUnderPressure (OperationContext *opCtx) const
 Indicates whether the storage engine cache is under pressure. More...
 
virtual void setCachePressureForTest (int pressure)
 For unit tests only. More...
 
virtual void replicationBatchIsComplete () const
 Notifies the storage engine that a replication batch has completed. More...
 
virtual StatusWith< std::vector< CollectionIndexNamePair > > reconcileCatalogAndIdents (OperationContext *opCtx)
 Drop abandoned idents. More...
 
virtual Timestamp getAllCommittedTimestamp () const =0
 Returns the all committed timestamp. More...
 

Member Typedef Documentation

◆ CollectionIndexNamePair

typedef std::pair<std::string, std::string> mongo::StorageEngine::CollectionIndexNamePair

Constructor & Destructor Documentation

◆ ~StorageEngine()

virtual mongo::StorageEngine::~StorageEngine ( )
inlinevirtual

The destructor should only be called if we are tearing down but not exiting the process.

Member Function Documentation

◆ beginBackup()

virtual Status mongo::StorageEngine::beginBackup ( OperationContext *  opCtx)
inlinevirtual

Transitions the storage engine into backup mode.

During backup mode the storage engine must stabilize its on-disk files, and avoid any internal processing that may involve file I/O, such as online compaction, so a filesystem level backup may be performed.

Storage engines that do not support this feature should use the default implementation. Storage engines that implement this must also implement endBackup().

For Storage engines that implement beginBackup the _inBackupMode variable is provided to avoid multiple instance enterting/leaving backup concurrently.

If this function returns an OK status, MongoDB can call endBackup to signal the storage engine that filesystem writes may continue. This function should return a non-OK status if filesystem changes cannot be stopped to allow for online backup. If the function should be retried, returns a non-OK status. This function may throw a WriteConflictException, which should trigger a retry by the caller. All other exceptions should be treated as errors.

Reimplemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ cleanShutdown()

virtual void mongo::StorageEngine::cleanShutdown ( )
pure virtual

This method will be called before there is a clean shutdown.

Storage engines should override this method if they have clean-up to do that is different from unclean shutdown. MongoDB will not call into the storage subsystem after calling this function.

On error, the storage engine should assert and crash. There is intentionally no uncleanShutdown().

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ closeCatalog()

virtual void mongo::StorageEngine::closeCatalog ( OperationContext *  opCtx)
inlinevirtual

Reimplemented in mongo::KVStorageEngine.

◆ closeDatabase()

virtual Status mongo::StorageEngine::closeDatabase ( OperationContext *  opCtx,
StringData  db 
)
pure virtual

Closes all file handles associated with a database.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ dropDatabase()

virtual Status mongo::StorageEngine::dropDatabase ( OperationContext *  opCtx,
StringData  db 
)
pure virtual

Deletes all data and metadata for a database.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ endBackup()

virtual void mongo::StorageEngine::endBackup ( OperationContext *  opCtx)
inlinevirtual

Transitions the storage engine out of backup mode.

Storage engines that do not support this feature should use the default implementation.

Storage engines implementing this feature should fassert when unable to leave backup mode.

Reimplemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ finishInit()

virtual void mongo::StorageEngine::finishInit ( )
inlinevirtual

Called after the globalStorageEngine pointer has been set up, before any other methods are called.

Any initialization work that requires the ability to create OperationContexts should be done here rather than in the constructor.

Reimplemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ flushAllFiles()

virtual int mongo::StorageEngine::flushAllFiles ( OperationContext *  opCtx,
bool  sync 
)
pure virtual
Returns
number of files flushed

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ getAllCommittedTimestamp()

virtual Timestamp mongo::StorageEngine::getAllCommittedTimestamp ( ) const
pure virtual

Returns the all committed timestamp.

All transactions with timestamps earlier than the all committed timestamp are committed. Only storage engines that support document level locking must provide an implementation. Other storage engines may provide a no-op implementation.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ getDatabaseCatalogEntry()

virtual DatabaseCatalogEntry* mongo::StorageEngine::getDatabaseCatalogEntry ( OperationContext *  opCtx,
StringData  db 
)
pure virtual

Return the DatabaseCatalogEntry that describes the database indicated by 'db'.

StorageEngine owns returned pointer. It should not be deleted by any caller.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ getLastStableCheckpointTimestamp()

virtual boost::optional<Timestamp> mongo::StorageEngine::getLastStableCheckpointTimestamp ( ) const
inlinevirtual

Returns a timestamp that is guaranteed to be persisted to disk in a checkpoint.

Returns boost::none if there is no stable checkpoint. This method should return at least the value of getRecoveryTimestamp if the node started from a stable checkpoint. fasserts if StorageEngine::supportsRecoverToStableTimestamp() would return false.

Reimplemented in mongo::KVStorageEngine.

◆ getRecoveryTimestamp()

virtual boost::optional<Timestamp> mongo::StorageEngine::getRecoveryTimestamp ( ) const
inlinevirtual

Returns the stable timestamp that the storage engine recovered to on startup.

If the recovery point was not stable, returns "none". fasserts if StorageEngine::supportsRecoverToStableTimestamp() would return false.

Reimplemented in mongo::KVStorageEngine.

◆ getSnapshotManager()

virtual SnapshotManager* mongo::StorageEngine::getSnapshotManager ( ) const
inlinevirtual

Returns the SnapshotManager for this StorageEngine or NULL if not supported.

Pointer remains owned by the StorageEngine, not the caller.

Reimplemented in mongo::KVStorageEngine.

◆ isCacheUnderPressure()

virtual bool mongo::StorageEngine::isCacheUnderPressure ( OperationContext *  opCtx) const
inlinevirtual

Indicates whether the storage engine cache is under pressure.

Retrieves a cache pressure value in the range [0, 100] from the storage engine, and compares it against storageGlobalParams.cachePressureThreshold, a dynamic server parameter, to determine whether cache pressure is too high.

Reimplemented in mongo::KVStorageEngine.

◆ isDurable()

virtual bool mongo::StorageEngine::isDurable ( ) const
pure virtual

Returns whether the engine supports a journalling concept or not.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ isEphemeral()

virtual bool mongo::StorageEngine::isEphemeral ( ) const
pure virtual

Returns true if the engine does not persist data to disk; false otherwise.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ isMmapV1()

virtual bool mongo::StorageEngine::isMmapV1 ( ) const
inlinevirtual

Only MMAPv1 should override this and return true to trigger MMAPv1-specific behavior.

Reimplemented in mongo::MMAPV1Engine.

◆ listDatabases()

virtual void mongo::StorageEngine::listDatabases ( std::vector< std::string > *  out) const
pure virtual

List the databases stored in this storage engine.

XXX: why doesn't this take OpCtx?

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ loadCatalog()

virtual void mongo::StorageEngine::loadCatalog ( OperationContext *  opCtx)
inlinevirtual

Populates and tears down in-memory data structures, respectively.

Only required for storage engines that support recoverToStableTimestamp().

Must be called with the global lock acquired in exclusive mode.

Reimplemented in mongo::KVStorageEngine.

◆ newRecoveryUnit()

virtual RecoveryUnit* mongo::StorageEngine::newRecoveryUnit ( )
pure virtual

Returns a new interface to the storage engine's recovery unit.

The recovery unit is the durability interface. For details, see recovery_unit.h

Caller owns the returned pointer.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ reconcileCatalogAndIdents()

virtual StatusWith<std::vector<CollectionIndexNamePair> > mongo::StorageEngine::reconcileCatalogAndIdents ( OperationContext *  opCtx)
inlinevirtual

Drop abandoned idents.

In the successful case, returns a list of collection, index name pairs to rebuild.

Reimplemented in mongo::KVStorageEngine.

◆ recoverToStableTimestamp()

virtual StatusWith<Timestamp> mongo::StorageEngine::recoverToStableTimestamp ( OperationContext *  opCtx)
inlinevirtual

Recovers the storage engine state to the last stable timestamp.

"Stable" in this case refers to a timestamp that is guaranteed to never be rolled back. The stable timestamp used should be one provided by StorageEngine::setStableTimestamp().

The "local" database is exempt and should not roll back any state except for "local.replset.minvalid" which must roll back to the last stable timestamp.

If successful, returns the timestamp that the storage engine recovered to.

fasserts if StorageEngine::supportsRecoverToStableTimestamp() would return false. Returns a bad status if there is no stable timestamp to recover to.

It is illegal to call this concurrently with setStableTimestamp or setInitialDataTimestamp.

Reimplemented in mongo::KVStorageEngine.

◆ repairRecordStore()

virtual Status mongo::StorageEngine::repairRecordStore ( OperationContext *  opCtx,
const std::string &  ns 
)
pure virtual

Recover as much data as possible from a potentially corrupt RecordStore.

This only recovers the record data, not indexes or anything else.

Generally, this method should not be called directly except by the repairDatabase() free function.

NOTE: MMAPv1 does not support this method and has its own repairDatabase() method.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ replicationBatchIsComplete()

virtual void mongo::StorageEngine::replicationBatchIsComplete ( ) const
inlinevirtual

Notifies the storage engine that a replication batch has completed.

This means that all the writes associated with the oplog entries in the batch are finished and no new writes with timestamps associated with those oplog entries will show up in the future. This function can be used to ensure oplog visibility rules are not broken, for example.

Reimplemented in mongo::KVStorageEngine.

◆ setCachePressureForTest()

virtual void mongo::StorageEngine::setCachePressureForTest ( int  pressure)
inlinevirtual

For unit tests only.

Sets the cache pressure value with which isCacheUnderPressure() evalutates to 'pressure'.

Reimplemented in mongo::KVStorageEngine.

◆ setInitialDataTimestamp()

virtual void mongo::StorageEngine::setInitialDataTimestamp ( Timestamp  timestamp)
inlinevirtual

Tells the storage engine the timestamp of the data at startup.

This is necessary because timestamps are not persisted in the storage layer.

Reimplemented in mongo::KVStorageEngine.

◆ setJournalListener()

virtual void mongo::StorageEngine::setJournalListener ( JournalListener *  jl)
pure virtual

Sets a new JournalListener, which is used by the storage engine to alert the rest of the system about journaled write progress.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ setOldestTimestamp()

virtual void mongo::StorageEngine::setOldestTimestamp ( Timestamp  timestamp)
inlinevirtual

Sets the oldest timestamp for which the storage engine must maintain snapshot history through.

Additionally, all future writes must be newer or equal to this value.

Reimplemented in mongo::KVStorageEngine.

◆ setOldestTimestampFromStable()

virtual void mongo::StorageEngine::setOldestTimestampFromStable ( )
inlinevirtual

Uses the current stable timestamp to set the oldest timestamp for which the storage engine must maintain snapshot history through.

oldest_timestamp will be set to stable_timestamp adjusted by 'targetSnapshotHistoryWindowInSeconds' to create a window of available snapshots on the storage engine from oldest to stable. Furthermore, oldest_timestamp will never be set ahead of the oplog read timestamp, ensuring the oplog reader's 'read_timestamp' can always be serviced.

Reimplemented in mongo::KVStorageEngine.

◆ setStableTimestamp()

virtual void mongo::StorageEngine::setStableTimestamp ( Timestamp  timestamp)
inlinevirtual

Sets the highest timestamp at which the storage engine is allowed to take a checkpoint.

This timestamp can never decrease, and thus should be a timestamp that can never roll back.

Reimplemented in mongo::KVStorageEngine.

◆ supportsCappedCollections()

virtual bool mongo::StorageEngine::supportsCappedCollections ( ) const
inlinevirtual

Returns whether the storage engine supports capped collections.

Reimplemented in mongo::KVStorageEngine.

◆ supportsDBLocking()

virtual bool mongo::StorageEngine::supportsDBLocking ( ) const
inlinevirtual

Returns whether the storage engine supports locking at a database level.

Reimplemented in mongo::KVStorageEngine.

◆ supportsDocLocking()

virtual bool mongo::StorageEngine::supportsDocLocking ( ) const
pure virtual

Returns whether the storage engine supports its own locking locking below the collection level.

If the engine returns true, MongoDB will acquire intent locks down to the collection level and will assume that the engine will ensure consistency at the level of documents. If false, MongoDB will lock the entire collection in Shared/Exclusive mode for read/write operations respectively.

Implemented in mongo::KVStorageEngine, and mongo::MMAPV1Engine.

◆ supportsReadConcernSnapshot()

virtual bool mongo::StorageEngine::supportsReadConcernSnapshot ( ) const
inlinevirtual

Returns true if the storage engine supports the readConcern level "snapshot".

Reimplemented in mongo::KVStorageEngine.

◆ supportsRecoverToStableTimestamp()

virtual bool mongo::StorageEngine::supportsRecoverToStableTimestamp ( ) const
inlinevirtual

Returns whether the storage engine supports "recover to stable timestamp".

Returns true if the storage engine supports "recover to stable timestamp" but does not currently have a stable timestamp. In that case StorageEngine::recoverToStableTimestamp() will return a bad status.

Reimplemented in mongo::KVStorageEngine.

The documentation for this class was generated from the following file: