org.exist.storage
Class NativeBroker

java.lang.Object
  java.util.Observable
      org.exist.storage.DBBroker
          org.exist.storage.NativeBroker

Direct Known Subclasses:

NativeClusterBroker

public class NativeBroker
extends DBBroker

Main class for the native XML storage backend. By "native" it is meant file-based, embedded backend. Provides access to all low-level operations required by the database. Extends DBBroker. Observer Design Pattern: role : this class is the subject (alias observable) for various classes that generate indices for the database content :

Author:

Wolfgang Meier

Nested Class Summary

static class

NativeBroker.NodeRef

Field Summary

static int	BUFFERS default buffer size setting
static byte	COLLECTIONS_DBX_ID
static String	DEFAULT_DATA_DIR
static int	DEFAULT_INDEX_DEPTH
static int	DEFAULT_MIN_MEMORY
static int	DEFAULT_NODES_BEFORE_MEMORY_CHECK check available memory after storing DEFAULT_NODES_BEFORE_MEMORY_CHECK nodes
static int	DEFAULT_PAGE_SIZE
static byte	DOM_DBX_ID
static byte	ELEMENTS_DBX_ID
static int	OFFSET_COLLECTION_ID
static int	OFFSET_VALUE
static byte	PREPEND_DB_ALWAYS
static byte	PREPEND_DB_AS_NEEDED
static byte	PREPEND_DB_NEVER
static String	PROPERTY_INDEX_DEPTH
static String	PROPERTY_MIN_FREE_MEMORY
static String	PROPERTY_PAGE_SIZE
static long	TEMP_FRAGMENT_TIMEOUT
static byte	VALUES_DBX_ID
static byte	WORDS_DBX_ID

Fields inherited from class org.exist.storage.DBBroker

customProperties, MATCH_EXACT, MATCH_REGEXP, MATCH_WILDCARDS, PROPERTY_XUPDATE_CONSISTENCY_CHECKS, PROPERTY_XUPDATE_FRAGMENTATION_FACTOR, PROPERTY_XUPDATE_GROWTH_FACTOR, ROOT_COLLECTION, ROOT_COLLECTION_NAME, SYSTEM_COLLECTION, TEMP_COLLECTION

Constructor Summary

NativeBroker(BrokerPool pool, Configuration config)
initialize database; read configuration, etc.

Method Summary

void	addObserver(Observer o)
void	checkXMLResourceConsistency(DocumentImpl doc) consistency Check of the database; useful after XUpdates; called if xupdate.consistency-checks is true in configuration
void	checkXMLResourceTree(DocumentImpl doc) consistency Check of the database; useful after XUpdates; called by checkXMLResourceConsistency(DocumentImpl)
void	cleanUpTempCollection() remove temporary collection
void	cleanUpTempResources() remove all documents from temporary collection
void	closeDocument()
void	copyCollection(Txn transaction, Collection collection, Collection destination, XmldbURI newUri) Copy a collection to the destination collection and rename it.
void	copyXMLResource(Txn transaction, DocumentImpl doc, Collection destination, XmldbURI newName) Copy a resource to the destination collection and rename it.
void	defragXMLResource(Txn transaction, DocumentImpl doc) Defragment pages of this document.
void	deleteObservers()
void	dropCollectionIndex(Collection collection)
void	endElement(StoredNode node, NodePath currentPath, String content, boolean remove) Update indexes for the given element node.
void	endRemove(Txn transaction) Takes care of actually remove entries from the indices; must be called after one or more call to removeNode(Txn, StoredNode, NodePath, String).
void	flush() Flush all data that has not been written before.
DocumentSet	getAllXMLResources(DocumentSet docs) Adds all the documents in the database to the specified DocumentSet.
byte[]	getBinaryResource(BinaryDocument blob) Retrieve the binary data stored under the resource descriptor BinaryDocument.
Collection	getCollection(XmldbURI uri) Returns the database collection identified by the specified path.
void	getCollectionResources(Collection collection)
DOMFile	getDOMFile()
ElementIndex	getElementIndex()
short	getFreeCollectionId(Txn transaction) Get the next free collection id.
int	getFreeResourceId(Txn transaction) Get the next unused document id.
IndexSpec	getIndexConfiguration()
short	getNextCollectionId(Txn transaction) Get the next available unique collection id.
int	getNextResourceId(Txn transaction, Collection collection) get next Free Doc Id
Iterator	getNodeIterator(StoredNode node) Return a NodeIterator starting at the specified node.
String	getNodeValue(StoredNode node, boolean addWhitespace) Get the string value of the specified node.
Collection	getOrCreateCollection(Txn transaction, XmldbURI name) Returns the database collection identified by the specified path.
int	getPageSize()
void	getResourceMetadata(DocumentImpl document)
Serializer	getSerializer() Get an instance of the Serializer used for converting nodes back to XML.
BTree	getStorage(byte id)
byte[]	getStorageFileIds()
SymbolTable	getSymbols()
TextSearchEngine	getTextEngine() Get the TextSearchEngine associated with this broker.
NativeValueIndex	getValueIndex()
Document	getXMLResource(XmldbURI fileName) get a document by its file name.
DocumentImpl	getXMLResource(XmldbURI fileName, int lockMode) Return the document stored at the specified path.
DocumentSet	getXMLResourcesByDoctype(String doctypeName, DocumentSet result) Get all the documents in this database matching the given document-type's name.
EmbeddedXMLStreamReader	getXMLStreamReader(NodeProxy proxy, boolean reportAttributes)
EmbeddedXMLStreamReader	getXMLStreamReader(StoredNode node, boolean reportAttributes)
void	indexNode(Txn transaction, StoredNode node, NodePath currentPath) Index a single node, which has been added through an XUpdate operation.
void	indexNode(Txn transaction, StoredNode node, NodePath currentPath, int repairMode)
void	insertNodeAfter(Txn transaction, StoredNode previous, StoredNode node) Physically insert a node into the DOM storage.
boolean	isReadOnly() Is the database running read-only? Returns false by default.
void	moveCollection(Txn transaction, Collection collection, Collection destination, XmldbURI newName) Move a collection and all its subcollections to another collection and rename it.
void	moveXMLResource(Txn transaction, DocumentImpl doc, Collection destination, XmldbURI newName) move Resource to another collection, with possible rename
Serializer	newSerializer()
StoredNode	objectWith(Document doc, NodeId nodeId) Get a node with given owner document and id from the database.
StoredNode	objectWith(NodeProxy p)
Collection	openCollection(XmldbURI uri, int lockMode) Open a collection for reading or writing.
XmldbURI	prepend(XmldbURI uri)
void	readBinaryResource(BinaryDocument blob, OutputStream os)
void	reindexCollection(Collection collection, int mode)
void	reindexCollection(Txn transaction, Collection collection, int mode)
void	reindexCollection(XmldbURI collectionName) Reindex a collection.
void	removeAllNodes(Txn transaction, StoredNode node, NodePath currentPath, StreamListener listener)
void	removeBinaryResource(Txn transaction, BinaryDocument blob) Completely delete this binary document (descriptor and binary data).
boolean	removeCollection(Txn transaction, Collection collection) Remove the collection and all its subcollections from the database.
void	removeNode(Txn transaction, StoredNode node, NodePath currentPath, String content) Removes the Node Reference from the database.
void	removeXMLResource(Txn transaction, DocumentImpl document, boolean freeDocId)
void	repair()
void	saveCollection(Txn transaction, Collection collection) Saves the specified collection to storage.
void	shutdown() Shut down the database instance.
void	storeBinaryResource(Txn transaction, BinaryDocument blob, byte[] data) Stores the given data under the given binary resource descriptor (BinaryDocument).
void	storeBinaryResource(Txn transaction, BinaryDocument blob, InputStream is) Stores the given data under the given binary resource descriptor (BinaryDocument).
void	storeNode(Txn transaction, StoredNode node, NodePath currentPath)
void	storeNode(Txn transaction, StoredNode node, NodePath currentPath, boolean fullTextIndex) Store a node into the database.
DocumentImpl	storeTempResource(DocumentImpl doc) store into the temporary collection of the database a given in-memory Document
void	storeXMLResource(Txn transaction, DocumentImpl doc) store Document entry into its collection.
void	sync(int syncEvent) Sync dom and collection state data (pages) to disk.
void	updateNode(Txn transaction, StoredNode node, boolean reindex) Update a node's data.

Methods inherited from class org.exist.storage.DBBroker

addContentLoadingObserver, clearContentLoadingObservers, decReferenceCount, endElement, getBrokerPool, getConfiguration, getId, getIndexController, getReferenceCount, getUser, getXQueryService, incReferenceCount, indexNode, removeContentLoadingObserver, removeXMLResource, setId, setUser, toString

Methods inherited from class java.util.Observable

countObservers, deleteObserver, hasChanged, notifyObservers, notifyObservers

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

PREPEND_DB_ALWAYS

public static final byte PREPEND_DB_ALWAYS

See Also:

Constant Field Values

PREPEND_DB_NEVER

public static final byte PREPEND_DB_NEVER

See Also:

Constant Field Values

PREPEND_DB_AS_NEEDED

public static final byte PREPEND_DB_AS_NEEDED

See Also:

Constant Field Values

COLLECTIONS_DBX_ID

public static final byte COLLECTIONS_DBX_ID

See Also:

Constant Field Values

ELEMENTS_DBX_ID

public static final byte ELEMENTS_DBX_ID

See Also:

Constant Field Values

VALUES_DBX_ID

public static final byte VALUES_DBX_ID

See Also:

Constant Field Values

WORDS_DBX_ID

public static final byte WORDS_DBX_ID

See Also:

Constant Field Values

DOM_DBX_ID

public static final byte DOM_DBX_ID

See Also:

Constant Field Values

PROPERTY_PAGE_SIZE

public static final String PROPERTY_PAGE_SIZE

See Also:

Constant Field Values

PROPERTY_MIN_FREE_MEMORY

public static final String PROPERTY_MIN_FREE_MEMORY

See Also:

Constant Field Values

PROPERTY_INDEX_DEPTH

public static final String PROPERTY_INDEX_DEPTH

See Also:

Constant Field Values

DEFAULT_DATA_DIR

public static final String DEFAULT_DATA_DIR

See Also:

Constant Field Values

DEFAULT_PAGE_SIZE

public static final int DEFAULT_PAGE_SIZE

See Also:

Constant Field Values

DEFAULT_INDEX_DEPTH

public static final int DEFAULT_INDEX_DEPTH

See Also:

Constant Field Values

DEFAULT_MIN_MEMORY

public static final int DEFAULT_MIN_MEMORY

See Also:

Constant Field Values

TEMP_FRAGMENT_TIMEOUT

public static final long TEMP_FRAGMENT_TIMEOUT

See Also:

Constant Field Values

BUFFERS

public static final int BUFFERS

default buffer size setting

See Also:

Constant Field Values

DEFAULT_NODES_BEFORE_MEMORY_CHECK

public static final int DEFAULT_NODES_BEFORE_MEMORY_CHECK

check available memory after storing DEFAULT_NODES_BEFORE_MEMORY_CHECK nodes

See Also:

Constant Field Values

OFFSET_COLLECTION_ID

public static int OFFSET_COLLECTION_ID

OFFSET_VALUE

public static int OFFSET_VALUE

Constructor Detail

NativeBroker

public NativeBroker(BrokerPool pool,
                    Configuration config)
             throws EXistException

initialize database; read configuration, etc.

Throws:

EXistException

Method Detail

addObserver

public void addObserver(Observer o)

Overrides:

addObserver in class Observable

deleteObservers

public void deleteObservers()

Overrides:

deleteObservers in class Observable

endElement

public void endElement(StoredNode node,
                       NodePath currentPath,
                       String content,
                       boolean remove)

Update indexes for the given element node. This method is called when the indexer encounters a closing element tag. It updates any range indexes defined on the element value and adds the element id to the structural index.

Specified by:

endElement in class DBBroker

Parameters:

node - the current element node

currentPath - node path leading to the element

content - contains the string value of the element. Needed if a range index is defined on it.

endRemove

public void endRemove(Txn transaction)

Takes care of actually remove entries from the indices; must be called after one or more call to removeNode(Txn, StoredNode, NodePath, String).

Specified by:

endRemove in class DBBroker

isReadOnly

public boolean isReadOnly()

Description copied from class: DBBroker

Is the database running read-only? Returns false by default. Storage backends should override this if they support read-only mode.

Overrides:

isReadOnly in class DBBroker

Returns:

boolean

getPageSize

public int getPageSize()

Specified by:

getPageSize in class DBBroker

getDOMFile

public DOMFile getDOMFile()

getStorage

public BTree getStorage(byte id)

getStorageFileIds

public byte[] getStorageFileIds()

getSymbols

public SymbolTable getSymbols()

Specified by:

getSymbols in class DBBroker

getIndexConfiguration

public IndexSpec getIndexConfiguration()

Specified by:

getIndexConfiguration in class DBBroker

getElementIndex

public ElementIndex getElementIndex()

Specified by:

getElementIndex in class DBBroker

getValueIndex

public NativeValueIndex getValueIndex()

Specified by:

getValueIndex in class DBBroker

getTextEngine

public TextSearchEngine getTextEngine()

Description copied from class: DBBroker

Get the TextSearchEngine associated with this broker. Every subclass of DBBroker will have it's own implementation of TextSearchEngine.

Specified by:

getTextEngine in class DBBroker

getXMLStreamReader

public EmbeddedXMLStreamReader getXMLStreamReader(StoredNode node,
                                                  boolean reportAttributes)
                                           throws IOException,
                                                  javax.xml.stream.XMLStreamException

Specified by:

getXMLStreamReader in class DBBroker

Throws:

IOException

javax.xml.stream.XMLStreamException

getXMLStreamReader

public EmbeddedXMLStreamReader getXMLStreamReader(NodeProxy proxy,
                                                  boolean reportAttributes)
                                           throws IOException,
                                                  javax.xml.stream.XMLStreamException

Specified by:

getXMLStreamReader in class DBBroker

Throws:

IOException

javax.xml.stream.XMLStreamException

getNodeIterator

public Iterator getNodeIterator(StoredNode node)

Description copied from class: DBBroker

Return a NodeIterator starting at the specified node.

Overrides:

getNodeIterator in class DBBroker

Returns:

NodeIterator of node.

getSerializer

public Serializer getSerializer()

Description copied from class: DBBroker

Get an instance of the Serializer used for converting nodes back to XML. Subclasses of DBBroker may have specialized subclasses of Serializer to convert a node into an XML-string

Specified by:

getSerializer in class DBBroker

newSerializer

public Serializer newSerializer()

Specified by:

newSerializer in class DBBroker

prepend

public XmldbURI prepend(XmldbURI uri)

cleanUpTempCollection

public void cleanUpTempCollection()
                           throws IOException

remove temporary collection

Specified by:

cleanUpTempCollection in class DBBroker

Throws:

IOException

getOrCreateCollection

public Collection getOrCreateCollection(Txn transaction,
                                        XmldbURI name)
                                 throws PermissionDeniedException,
                                        IOException

Description copied from class: DBBroker

Returns the database collection identified by the specified path. If the collection does not yet exist, it is created - including all ancestors. The path should be absolute, e.g. /db/shakespeare.

Specified by:

getOrCreateCollection in class DBBroker

Parameters:

transaction - The transaction, which registers the acquired write locks. The locks should be released on commit/abort.

name - The collection's URI

Returns:

The collection or null if no collection matches the path

Throws:

PermissionDeniedException

IOException

getCollection

public Collection getCollection(XmldbURI uri)

Description copied from class: DBBroker

Returns the database collection identified by the specified path. The path should be absolute, e.g. /db/shakespeare.

Specified by:

getCollection in class DBBroker

Returns:

collection or null if no collection matches the path

openCollection

public Collection openCollection(XmldbURI uri,
                                 int lockMode)

Description copied from class: DBBroker

Open a collection for reading or writing. The collection is identified by its absolute path, e.g. /db/shakespeare. It will be loaded and locked according to the lockMode argument. The caller should take care to release the collection lock properly.

Specified by:

openCollection in class DBBroker

Parameters:

uri - The collection path

lockMode - one of the modes specified in class Lock

Returns:

collection or null if no collection matches the path

copyCollection

public void copyCollection(Txn transaction,
                           Collection collection,
                           Collection destination,
                           XmldbURI newUri)
                    throws PermissionDeniedException,
                           LockException,
                           IOException

Description copied from class: DBBroker

Copy a collection to the destination collection and rename it.

Specified by:

copyCollection in class DBBroker

collection - the resource to move

destination - the destination collection

newUri - the new name the resource should have in the destination collection

Throws:

PermissionDeniedException

LockException

IOException

moveCollection

public void moveCollection(Txn transaction,
                           Collection collection,
                           Collection destination,
                           XmldbURI newName)
                    throws PermissionDeniedException,
                           LockException,
                           IOException

Description copied from class: DBBroker

Move a collection and all its subcollections to another collection and rename it. Moving a collection just modifies the collection path and all resource paths. The data itself remains in place.

Specified by:

moveCollection in class DBBroker

collection - the collection to move

destination - the destination collection

newName - the new name the collection should have in the destination collection

Throws:

PermissionDeniedException

LockException

IOException

removeCollection

public boolean removeCollection(Txn transaction,
                                Collection collection)
                         throws PermissionDeniedException,
                                IOException

Description copied from class: DBBroker

Remove the collection and all its subcollections from the database.

Specified by:

removeCollection in class DBBroker

Throws:

PermissionDeniedException

IOException

saveCollection

public void saveCollection(Txn transaction,
                           Collection collection)
                    throws PermissionDeniedException,
                           IOException

Saves the specified collection to storage. Collections are usually cached in memory. If a collection is modified, this method needs to be called to make the changes persistent. Note: appending a new document to a collection does not require a save.

Specified by:

saveCollection in class DBBroker

collection - Collection to store

Throws:

PermissionDeniedException

IOException

getFreeCollectionId

public short getFreeCollectionId(Txn transaction)
                          throws ReadOnlyException

Get the next free collection id. If a collection is removed, its collection id is released so it can be reused.

Returns:

next free collection id.

Throws:

ReadOnlyException

getNextCollectionId

public short getNextCollectionId(Txn transaction)
                          throws ReadOnlyException

Get the next available unique collection id.

Returns:

next available unique collection id

Throws:

ReadOnlyException

reindexCollection

public void reindexCollection(XmldbURI collectionName)
                       throws PermissionDeniedException

Description copied from class: DBBroker

Reindex a collection.

Specified by:

reindexCollection in class DBBroker

Throws:

PermissionDeniedException - public abstract void reindexCollection(String collectionName) throws PermissionDeniedException;

reindexCollection

public void reindexCollection(Collection collection,
                              int mode)
                       throws PermissionDeniedException

Throws:

PermissionDeniedException

reindexCollection

public void reindexCollection(Txn transaction,
                              Collection collection,
                              int mode)
                       throws PermissionDeniedException

Throws:

PermissionDeniedException

dropCollectionIndex

public void dropCollectionIndex(Collection collection)
                         throws PermissionDeniedException

Throws:

PermissionDeniedException

storeTempResource

public DocumentImpl storeTempResource(DocumentImpl doc)
                               throws EXistException,
                                      PermissionDeniedException,
                                      LockException

store into the temporary collection of the database a given in-memory Document

Specified by:

storeTempResource in class DBBroker

Throws:

EXistException

PermissionDeniedException

LockException

cleanUpTempResources

public void cleanUpTempResources()

remove all documents from temporary collection

Specified by:

cleanUpTempResources in class DBBroker

storeXMLResource

public void storeXMLResource(Txn transaction,
                             DocumentImpl doc)

store Document entry into its collection.

Specified by:

storeXMLResource in class DBBroker

doc - the document's metadata to store.

storeBinaryResource

public void storeBinaryResource(Txn transaction,
                                BinaryDocument blob,
                                byte[] data)

Description copied from class: DBBroker

Stores the given data under the given binary resource descriptor (BinaryDocument).

Specified by:

storeBinaryResource in class DBBroker

blob - the binary document descriptor

data - the document binary data

storeBinaryResource

public void storeBinaryResource(Txn transaction,
                                BinaryDocument blob,
                                InputStream is)

Description copied from class: DBBroker

Stores the given data under the given binary resource descriptor (BinaryDocument).

Specified by:

storeBinaryResource in class DBBroker

blob - the binary document descriptor

is - the document binary data as input stream

getXMLResource

public Document getXMLResource(XmldbURI fileName)
                        throws PermissionDeniedException

get a document by its file name. The document's file name is used to identify a document.

Specified by:

getXMLResource in class DBBroker

Parameters:

fileName - absolute file name in the database; name can be given with or without the leading path /db/shakespeare.

Returns:

The document value

Throws:

PermissionDeniedException

getXMLResource

public DocumentImpl getXMLResource(XmldbURI fileName,
                                   int lockMode)
                            throws PermissionDeniedException

Description copied from class: DBBroker

Return the document stored at the specified path. The path should be absolute, e.g. /db/shakespeare/plays/hamlet.xml, with the specified lock.

Specified by:

getXMLResource in class DBBroker

Returns:

the document or null if no document could be found at the specified location.

Throws:

PermissionDeniedException

getBinaryResource

public byte[] getBinaryResource(BinaryDocument blob)

Description copied from class: DBBroker

Retrieve the binary data stored under the resource descriptor BinaryDocument.

Specified by:

getBinaryResource in class DBBroker

Parameters:

blob - the binary document descriptor

Returns:

the document binary data

readBinaryResource

public void readBinaryResource(BinaryDocument blob,
                               OutputStream os)

Specified by:

readBinaryResource in class DBBroker

getCollectionResources

public void getCollectionResources(Collection collection)

Specified by:

getCollectionResources in class DBBroker

getXMLResourcesByDoctype

public DocumentSet getXMLResourcesByDoctype(String doctypeName,
                                            DocumentSet result)

Get all the documents in this database matching the given document-type's name.

Specified by:

getXMLResourcesByDoctype in class DBBroker

Returns:

The documentsByDoctype value

getAllXMLResources

public DocumentSet getAllXMLResources(DocumentSet docs)

Adds all the documents in the database to the specified DocumentSet.

Specified by:

getAllXMLResources in class DBBroker

Parameters:

docs - a (possibly empty) document set to which the found documents are added.

getResourceMetadata

public void getResourceMetadata(DocumentImpl document)

Specified by:

getResourceMetadata in class DBBroker

copyXMLResource

public void copyXMLResource(Txn transaction,
                            DocumentImpl doc,
                            Collection destination,
                            XmldbURI newName)
                     throws PermissionDeniedException,
                            LockException

Description copied from class: DBBroker

Copy a resource to the destination collection and rename it.

Specified by:

copyXMLResource in class DBBroker

doc - the resource to copy

destination - the destination collection

newName - the new name the resource should have in the destination collection

Throws:

PermissionDeniedException

LockException

moveXMLResource

public void moveXMLResource(Txn transaction,
                            DocumentImpl doc,
                            Collection destination,
                            XmldbURI newName)
                     throws PermissionDeniedException,
                            LockException,
                            IOException

move Resource to another collection, with possible rename

Specified by:

moveXMLResource in class DBBroker

doc - the resource to move

destination - the destination collection

newName - the new name the resource should have in the destination collection

Throws:

PermissionDeniedException

LockException

IOException

removeXMLResource

public void removeXMLResource(Txn transaction,
                              DocumentImpl document,
                              boolean freeDocId)
                       throws PermissionDeniedException

Specified by:

removeXMLResource in class DBBroker

Throws:

PermissionDeniedException

removeBinaryResource

public void removeBinaryResource(Txn transaction,