Class ZoneTree<TKey, TValue>

Inheritance

object

ZoneTree<TKey, TValue>

Implements

IZoneTree<TKey, TValue>

IDisposable

IZoneTreeMaintenance<TKey, TValue>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Tenray.ZoneTree.Core

Assembly: ZoneTree.dll

Syntax

public sealed class ZoneTree<TKey, TValue> : IZoneTree<TKey, TValue>, IDisposable, IZoneTreeMaintenance<TKey, TValue>

Type Parameters

Name	Description
TKey
TValue

Constructors

ZoneTree(ZoneTreeOptions<TKey, TValue>)

Declaration

public ZoneTree(ZoneTreeOptions<TKey, TValue> options)

Parameters

Type	Name	Description
ZoneTreeOptions<TKey, TValue>	options

ZoneTree(ZoneTreeOptions<TKey, TValue>, ZoneTreeMeta, IReadOnlyList<IReadOnlySegment<TKey, TValue>>, IMutableSegment<TKey, TValue>, IDiskSegment<TKey, TValue>, IReadOnlyList<IDiskSegment<TKey, TValue>>, long)

Declaration

public ZoneTree(ZoneTreeOptions<TKey, TValue> options, ZoneTreeMeta meta, IReadOnlyList<IReadOnlySegment<TKey, TValue>> readOnlySegments, IMutableSegment<TKey, TValue> mutableSegment, IDiskSegment<TKey, TValue> diskSegment, IReadOnlyList<IDiskSegment<TKey, TValue>> bottomSegments, long maximumSegmentId)

Parameters

Type	Name	Description
ZoneTreeOptions<TKey, TValue>	options
ZoneTreeMeta	meta
IReadOnlyList<IReadOnlySegment<TKey, TValue>>	readOnlySegments
IMutableSegment<TKey, TValue>	mutableSegment
IDiskSegment<TKey, TValue>	diskSegment
IReadOnlyList<IDiskSegment<TKey, TValue>>	bottomSegments
long	maximumSegmentId

Fields

SegmentWalCategory

Declaration

public const string SegmentWalCategory = "seg"

Field Value

Type	Description
string

Properties

BottomSegments

Gets current bottom segments.

Declaration

public IReadOnlyList<IDiskSegment<TKey, TValue>> BottomSegments { get; }

Property Value

Type	Description
IReadOnlyList<IDiskSegment<TKey, TValue>>

Comparer

The key comparer.

Declaration

public IRefComparer<TKey> Comparer { get; }

Property Value

Type	Description
IRefComparer<TKey>

DiskSegment

Gets current disk segment.

Declaration

public IDiskSegment<TKey, TValue> DiskSegment { get; }

Property Value

Type	Description
IDiskSegment<TKey, TValue>

InMemoryRecordCount

Retrieves the total number of records that lies in memory excluding the sparse array records of DiskSegment. In an LSM tree, records can be duplicated across different segments.

Declaration

public long InMemoryRecordCount { get; }

Property Value

Type	Description
long

IsBottomSegmentsMerging

true if bottom segments merge operation is running, otherwise false.

Declaration

public bool IsBottomSegmentsMerging { get; }

Property Value

Type	Description
bool

IsMerging

true if merge operation is running, otherwise false.

Declaration

public bool IsMerging { get; }

Property Value

Type	Description
bool

IsReadOnly

Enables read-only mode.

Declaration

public bool IsReadOnly { get; set; }

Property Value

Type	Description
bool

KeySerializer

The key serializer.

Declaration

public ISerializer<TKey> KeySerializer { get; }

Property Value

Type	Description
ISerializer<TKey>

Logger

ZoneTree Logger.

Declaration

public ILogger Logger { get; }

Property Value

Type	Description
ILogger

Maintenance

Returns maintenance object belongs to this ZoneTree.

Declaration

public IZoneTreeMaintenance<TKey, TValue> Maintenance { get; }

Property Value

Type	Description
IZoneTreeMaintenance<TKey, TValue>

MutableSegment

Gets current mutable segment. Mutable segment is the only writable part of the LSM tree.

Declaration

public IMutableSegment<TKey, TValue> MutableSegment { get; }

Property Value

Type	Description
IMutableSegment<TKey, TValue>

MutableSegmentRecordCount

Retrieves the number of records in mutable segment.

Declaration

public long MutableSegmentRecordCount { get; }

Property Value

Type	Description
long

ReadOnlySegments

Gets current readonly segments in-memory. MoveMutableSegmentForward operation moves writable segment to the read-only segments layer. The readonly segments remains in memory until merge operation done.

Declaration

public IReadOnlyList<IReadOnlySegment<TKey, TValue>> ReadOnlySegments { get; }

Property Value

Type	Description
IReadOnlyList<IReadOnlySegment<TKey, TValue>>

ReadOnlySegmentsCount

Retrieves the number of read only segments.

Declaration

public int ReadOnlySegmentsCount { get; }

Property Value

Type	Description
int

ReadOnlySegmentsRecordCount

Retrieves the number of records in read-only segments.

Declaration

public long ReadOnlySegmentsRecordCount { get; }

Property Value

Type	Description
long

TotalRecordCount

Retrieves the total number of records that lies in memory and disk excluding the sparse array records of DiskSegments. In an LSM tree, records can be duplicated across different segments. Hence, this is not the actual unique record count of the tree. To get exact record count, a partial database scan is needed. Use Count() and CountFullScan() for actual record count.

Declaration

public long TotalRecordCount { get; }

Property Value

Type	Description
long

ValueSerializer

The value serializer.

Declaration

public ISerializer<TValue> ValueSerializer { get; }

Property Value

Type	Description
ISerializer<TValue>

Methods

AtomicUpsert(in TKey, in TValue)

Adds or updates the specified key/value pair atomically across LSM-Tree segments.

Declaration

public void AtomicUpsert(in TKey key, in TValue value)

Parameters

Type	Name	Description
TKey	key	The key of the element to upsert.
TValue	value	The value of the element to upsert.

CollectSegments(bool, bool, bool)

Declaration

public ZoneTree<TKey, TValue>.SegmentCollection CollectSegments(bool includeMutableSegment, bool includeDiskSegment, bool includeBottomSegments)

Parameters

Type	Name	Description
bool	includeMutableSegment
bool	includeDiskSegment
bool	includeBottomSegments

Returns

Type	Description
ZoneTree<TKey, TValue>.SegmentCollection

ContainsKey(in TKey)

Checks the existence of the key in the tree.

Declaration

public bool ContainsKey(in TKey key)

Parameters

Type	Name	Description
TKey	key	The key of the element.

Returns

Type	Description
bool	true if key is found in tree; otherwise, false

Count()

Counts Keys in the entire database. This operation scans the in-memory segments and queries the disk segment.

Declaration

public long Count()

Returns

Type	Description
long	Number of the valid records in the tree.

CountFullScan()

Counts Keys in the entire database with a full scan.

Declaration

public long CountFullScan()

Returns

Type	Description
long	Number of the valid records in the tree.

Remarks

In regular cases, the disk segment does not contain deleted records. However, TTL or custom deletion logic would let the disk segment contains deleted records. In that case, a full scan is required for the count.

CreateInMemorySegmentsIterator(bool, bool)

Creates an iterator that enables scanning of the in-memory segments. This includes read-only segments and mutable segment.

Declaration

public IZoneTreeIterator<TKey, TValue> CreateInMemorySegmentsIterator(bool autoRefresh, bool includeDeletedRecords)

Parameters

Type	Name	Description
bool	autoRefresh	if true, auto refresh is enabled.
bool	includeDeletedRecords	if true the deleted records are included in iteration.

Returns

Type	Description
IZoneTreeIterator<TKey, TValue>	ZoneTree Iterator

CreateIterator(IteratorType, bool)

Creates an iterator that enables scanning of the entire database.

Declaration

public IZoneTreeIterator<TKey, TValue> CreateIterator(IteratorType iteratorType, bool includeDeletedRecords)

Parameters

Type	Name	Description
IteratorType	iteratorType	Defines iterator type.
bool	includeDeletedRecords	if true the iterator retrieves the deleted and normal records

Returns

Type	Description
IZoneTreeIterator<TKey, TValue>	ZoneTree Iterator

Remarks

The iterator might or might not retrieve newly inserted elements. This depends on the iterator's internal segment iterator positions.

If the newly inserted or deleted key is after the internal segment iterator position, the new data is included in the iteration.

Iterators are lightweight. Create them when you need and dispose them when you dont need. Iterators acquire locks on the disk segment and prevents its disposal.

Use snapshot iterators for consistent view by ignoring new writes.

CreateMaintainer()

Creates the default ZoneTree Maintainer.

Declaration

public IMaintainer CreateMaintainer()

Returns

Type	Description
IMaintainer

CreateReadOnlySegmentsIterator(bool, bool)

Creates an iterator that enables scanning of the readonly segments.

Declaration

public IZoneTreeIterator<TKey, TValue> CreateReadOnlySegmentsIterator(bool autoRefresh, bool includeDeletedRecords)

Parameters

Type	Name	Description
bool	autoRefresh
bool	includeDeletedRecords

Returns

Type	Description
IZoneTreeIterator<TKey, TValue>	ZoneTree Iterator

CreateReverseIterator(IteratorType, bool)

Creates a reverse iterator that enables scanning of the entire database.

Declaration

public IZoneTreeIterator<TKey, TValue> CreateReverseIterator(IteratorType iteratorType, bool includeDeletedRecords)

Parameters

Type	Name	Description
IteratorType	iteratorType	Defines iterator type.
bool	includeDeletedRecords	if true the iterator retrieves the deleted and normal records

Returns

Type	Description
IZoneTreeIterator<TKey, TValue>	ZoneTree Iterator

Remarks

ZoneTree iterator direction does not hurt performance. Forward and backward iterator's performances are equal.

DestroyTree()

Destroys the tree, deletes entire data and WAL store or folder.

Declaration

public void DestroyTree()

Dispose()

Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.

Declaration

public void Dispose()

ForceDelete(in TKey)

Deletes the specified key regardless of existence. (hint: LSM Tree delete is an insert) This is faster than TryDelete because it does not check existence in all layers. It increases the data lake size.

Declaration

public void ForceDelete(in TKey key)

Parameters

Type	Name	Description
TKey	key	The key of the element to delete.

MoveMutableSegmentForward()

Moves mutable segment into readonly segment. This will clear the writable region of the LSM tree. This method is thread safe and can be called from many threads.

Declaration

public void MoveMutableSegmentForward()

ReleaseCircularKeyCacheRecords()

Releases expired circular cache key records.

Declaration

public int ReleaseCircularKeyCacheRecords()

Returns

Type	Description
int	The total number of released cached records.

ReleaseCircularValueCacheRecords()

Releases expired circular cache value records.

Declaration

public int ReleaseCircularValueCacheRecords()

Returns

Type	Description
int	The total number of released cached records.

ReleaseReadBuffers(long)

Releases internal read buffers that have not been used since the specified tick count.

Declaration

public int ReleaseReadBuffers(long ticks)

Parameters

Type	Name	Description
long	ticks

Returns

Type	Description
int	The total number of released read buffers.

SaveMetaData()

Saves tree meta data and clears the meta wal record. After calling this method, the JSON meta file contains up to date tree meta data.

Saving meta file helps following:

1. Reduce the size of meta wal file.
2. Make Json meta file up to date to analyze parts of the LSM tree. Because Meta WAL file is not human readable.

It is up to user to decide when and how frequently save the meta file.

Declaration

public void SaveMetaData()

StartBottomSegmentsMergeOperation(int, int)

Merges selected bottom segments into a single bottom disk segment.

Declaration

public Thread StartBottomSegmentsMergeOperation(int fromIndex, int toIndex)

Parameters

Type	Name	Description
int	fromIndex	The lower bound
int	toIndex	The upper bound

Returns

Type	Description
Thread

StartMergeOperation()

Merges available in-memory read-only segments to the disk segment.

Declaration

public Thread StartMergeOperation()

Returns

Type	Description
Thread

TryAtomicAdd(in TKey, in TValue)

Attempts to add the specified key and value atomically across LSM-Tree segments.

Declaration

public bool TryAtomicAdd(in TKey key, in TValue value)

Parameters

Type	Name	Description
TKey	key	The key of the element to add.
TValue	value	The value of the element to add. It can be null.

Returns

Type	Description
bool	true if the key/value pair was added successfully; otherwise, false.

TryAtomicAddOrUpdate(in TKey, in TValue, ValueUpdaterDelegate<TValue>)

Attempts to add or update the specified key and value atomically across LSM-Tree segments.

Declaration

public bool TryAtomicAddOrUpdate(in TKey key, in TValue valueToAdd, ValueUpdaterDelegate<TValue> valueUpdater)

Parameters

Type	Name	Description
TKey	key	The key of the element to add.
TValue	valueToAdd	The value of the element to add. It can be null.
ValueUpdaterDelegate<TValue>	valueUpdater	The delegate function that updates the value.

Returns

Type	Description
bool	true if the key/value pair was added; false, if the key/value pair was updated.

TryAtomicGetAndUpdate(in TKey, out TValue, ValueUpdaterDelegate<TValue>)

Tries to get the value of the given key and updates the value atomically using value updater if found any.

Declaration

public bool TryAtomicGetAndUpdate(in TKey key, out TValue value, ValueUpdaterDelegate<TValue> valueUpdater)

Parameters

Type	Name	Description
TKey	key	The key of the element.
TValue	value	The value of the element associated with the key.
ValueUpdaterDelegate<TValue>	valueUpdater	The delegate function that updates the value.

Returns

Type	Description
bool	true if the key is found; otherwise, false

TryAtomicUpdate(in TKey, in TValue)

Attempts to update the specified key's value atomically across LSM-Tree segments.

Declaration

public bool TryAtomicUpdate(in TKey key, in TValue value)

Parameters

Type	Name	Description
TKey	key	The key of the element to update.
TValue	value	The value of the element to update. It can be null.

Returns

Type	Description
bool	true if the key/value pair was updated successfully; otherwise, false.

TryCancelBottomSegmentsMergeOperation()

Attempts to cancel bottom segments merge operation.

Declaration

public void TryCancelBottomSegmentsMergeOperation()

TryCancelMergeOperation()

Attempts to cancel merge operation.

Declaration

public void TryCancelMergeOperation()

TryDelete(in TKey)

Attempts to delete the specified key.

Declaration

public bool TryDelete(in TKey key)

Parameters

Type	Name	Description
TKey	key	The key of the element to delete.

Returns

Type	Description
bool	true if the key was found and deleted; false if the key was not found.

TryGet(in TKey, out TValue)

Tries to get the value of the given key.

Declaration

public bool TryGet(in TKey key, out TValue value)

Parameters

Type	Name	Description
TKey	key	The key of the element.
TValue	value	The value of the element associated with the key.

Returns

Type	Description
bool	true if the key is found; otherwise, false

TryGetAndUpdate(in TKey, out TValue, ValueUpdaterDelegate<TValue>)

Tries to get the value of the given key and updates the value using value updater if found any.

Declaration

public bool TryGetAndUpdate(in TKey key, out TValue value, ValueUpdaterDelegate<TValue> valueUpdater)

Parameters

Type	Name	Description
TKey	key	The key of the element.
TValue	value	The value of the element associated with the key.
ValueUpdaterDelegate<TValue>	valueUpdater	The delegate function that updates the value.

Returns

Type	Description
bool	true if the key is found; otherwise, false

Upsert(in TKey, in TValue)

Adds or updates the specified key/value pair.

Declaration

public void Upsert(in TKey key, in TValue value)

Parameters

Type	Name	Description
TKey	key	The key of the element to upsert.
TValue	value	The value of the element to upsert.

Remarks

This is a thread-safe method, but it is not sycnhronized with other atomic add/update/upsert methods. Using the Upsert method in parallel to atomic methods breaks the atomicity of the atomic methods.

For example: TryAtomicAddOrUpdate(key) does the following 3 things within lock to preserve atomicity across segments of LSM-Tree.

1. tries to get the value of the key
2. if it can find the key, it updates the value
3. if it cannot find the key, inserts the new key/value

All atomic methods respect this order using the same lock.

The Upsert method does not respect to the atomicity of atomic methods, because it does upsert without lock.

On the other hand, the Upsert method is atomic in the mutable segment scope but not across all segments. This makes Upsert method thread-safe.

This is the fastest add or update function.

Events

OnBottomSegmentsMergeOperationEnded

Event is fired when bottom segments merge operation is completed.

Declaration

public event BottomSegmentsMergeOperationEnded<TKey, TValue> OnBottomSegmentsMergeOperationEnded

Event Type

Type	Description
BottomSegmentsMergeOperationEnded<TKey, TValue>

OnBottomSegmentsMergeOperationStarted

Event is fired when bottom segments merge operation is completed.

Declaration

public event BottomSegmentsMergeOperationStarted<TKey, TValue> OnBottomSegmentsMergeOperationStarted

Event Type

Type	Description
BottomSegmentsMergeOperationStarted<TKey, TValue>

OnCanNotDropDiskSegment

Event is fired when a disk segment cannot be dropped. This does not harm the database consistency. The cleanup task can be done later.

Declaration

public event CanNotDropDiskSegment<TKey, TValue> OnCanNotDropDiskSegment

Event Type

Type	Description
CanNotDropDiskSegment<TKey, TValue>

OnCanNotDropDiskSegmentCreator

Event is fired when a disk segment creator cannot be dropped. This does not harm the database consistency. The cleanup task can be done later.

Declaration

public event CanNotDropDiskSegmentCreator<TKey, TValue> OnCanNotDropDiskSegmentCreator

Event Type

Type	Description
CanNotDropDiskSegmentCreator<TKey, TValue>

OnCanNotDropReadOnlySegment

Event is fired when a write ahead log cannot be dropped. This does not harm the database consistency. The cleanup task can be done later.

Declaration

public event CanNotDropReadOnlySegment<TKey, TValue> OnCanNotDropReadOnlySegment

Event Type

Type	Description
CanNotDropReadOnlySegment<TKey, TValue>

OnDiskSegmentActivated

Event is fired when the new disk segment is activated.

Declaration

public event DiskSegmentCreated<TKey, TValue> OnDiskSegmentActivated

Event Type

Type	Description
DiskSegmentCreated<TKey, TValue>

OnDiskSegmentCreated

Event is fired when the new disk segment is created. This is the best moment to initialize a sparse array. SparseArray is an in memory array that reduces disk reads. It is the best practice to call DiskSegment.InitSparseArray() when this event is fired.

Declaration

public event DiskSegmentCreated<TKey, TValue> OnDiskSegmentCreated

Event Type

Type	Description
DiskSegmentCreated<TKey, TValue>

OnMergeOperationEnded

Event is fired when merge operation is completed.

Declaration

public event MergeOperationEnded<TKey, TValue> OnMergeOperationEnded

Event Type

Type	Description
MergeOperationEnded<TKey, TValue>

OnMergeOperationStarted

Event is fired when merge operation is started.

Declaration

public event MergeOperationStarted<TKey, TValue> OnMergeOperationStarted

Event Type

Type	Description
MergeOperationStarted<TKey, TValue>

OnMutableSegmentMovedForward

Event is fired when mutable segment is moved forward.

Declaration

public event MutableSegmentMovedForward<TKey, TValue> OnMutableSegmentMovedForward

Event Type

Type	Description
MutableSegmentMovedForward<TKey, TValue>

OnZoneTreeIsDisposing

Event is fired when the ZoneTree is disposing.

Declaration

public event ZoneTreeIsDisposing<TKey, TValue> OnZoneTreeIsDisposing

Event Type

Type	Description
ZoneTreeIsDisposing<TKey, TValue>

Implements

IZoneTree<TKey, TValue>

IDisposable

IZoneTreeMaintenance<TKey, TValue>