state.backend.forst.cache.dir |
(none) |
String |
The directory where ForSt caches its SST files, fallback to the subdirectory of '/cache' under the value of 'state.backend.forst.local-dir' if not configured. |
state.backend.forst.cache.lru.access-before-promote |
6 |
Integer |
When the number of accesses to a block in cold link reaches this value, the block will be promoted to the head of the LRU list and become a hot link. The evicted file in cache will be reloaded as well. The default value is '5'. |
state.backend.forst.cache.lru.promote-limit |
3 |
Integer |
When the number of eviction that a block in hot link is moved to cold link reaches this value, the block will be blocked from being promoted to the head of the LRU list. The default value is '3'. |
state.backend.forst.cache.reserve-size |
256 mb |
MemorySize |
The amount of reserved size on disk space, and remaining space can be leveraged by the cache. The cache will evict the oldest files when the reserved space on disk (the disk where cache directory is) is not enough. User should specify at least one cache size limit to enable the cache, either this option or the 'state.backend.forst.cache.size-based-limit' option. They can be set simultaneously, and in this case, cache will grow if meet the requirements of both two options. If the specified file system of the cache directory does not support reading the remaining space, the cache will not be able to reserve the specified space, hence this option will be ignored. The default value is '256 mb', meaning the disk will be reserved that much space, and the remaining of the disk can be used for cache. A configured value of '0 bytes' means that this option is disabled. |
state.backend.forst.cache.size-based-limit |
0 bytes |
MemorySize |
An upper-bound of the size that can be used for cache. User should specify at least one cache size limit to enable the cache, either this option or the 'state.backend.forst.cache.reserve-size' option. They can be set simultaneously, and in this case, cache will grow if meet the requirements of both two options. The default value is '0 bytes', meaning that this option is disabled. |
state.backend.forst.executor.inline-coordinator |
false |
Boolean |
Whether to let the task thread be the coordinator thread responsible for distributing requests. If set to 'true', the task thread will be responsible for distributing requests, otherwise, a dedicated coordinator thread will be used. The default value is 'false'. |
state.backend.forst.executor.inline-write |
true |
Boolean |
Whether to let write requests be executed within the coordinator thread. If set to 'true', write requests will be executed within the coordinator thread, otherwise, a dedicated write thread will be used. The default value is 'true'. |
state.backend.forst.executor.read-io-parallelism |
3 |
Integer |
The number of threads used for read IO operations in the executor. |
state.backend.forst.executor.write-io-parallelism |
1 |
Integer |
The number of threads used for write IO operations in the executor. Only valid when 'state.backend.forst.executor.inline-write' is false. |
state.backend.forst.local-dir |
(none) |
String |
The local directory (on the TaskManager) where ForSt puts some metadata files. By default, it will be <WORKING_DIR>/tmp. See process.taskmanager.working-dir for more details. |
state.backend.forst.memory.fixed-per-slot |
(none) |
MemorySize |
The fixed total amount of memory per slot, shared among all ForSt instances.This option overrides the 'state.backend.forst.memory.managed' option. |
state.backend.forst.memory.fixed-per-tm |
(none) |
MemorySize |
The fixed total amount of memory per Task Manager, shared among all ForSt instances. This is a cluster-level option. This option only takes effect if 'state.backend.forst.memory.managed' is set to false and 'state.backend.forst.memory.fixed-per-slot' is not configured. If so, then each ForSt column family state has its own memory caches (as controlled by the column family options). The relevant options for the shared resources (e.g. write-buffer-ratio) can be set on the same level (config.yaml). Note that this feature breaks resource isolation between the slots. |
state.backend.forst.memory.high-prio-pool-ratio |
0.1 |
Double |
The fraction of cache memory that is reserved for high-priority data like index, filter, and compression dictionary blocks. This option only has an effect when 'state.backend.forst.memory.managed' or 'state.backend.forst.memory.fixed-per-slot' are configured. |
state.backend.forst.memory.managed |
true |
Boolean |
If set true, the ForSt state backend will automatically configure itself to use the managed memory budget of the task slot, and divide the memory over write buffers, indexes, block caches, etc. |
state.backend.forst.memory.partitioned-index-filters |
true |
Boolean |
With partitioning, the index/filter block of an SST file is partitioned into smaller blocks with an additional top-level index on them. When reading an index/filter, only top-level index is loaded into memory. The partitioned index/filter then uses the top-level index to load on demand into the block cache the partitions that are required to perform the index/filter query. This option only has an effect when 'state.backend.forst.memory.managed' or 'state.backend.forst.memory.fixed-per-slot' are configured. |
state.backend.forst.memory.write-buffer-ratio |
0.5 |
Double |
The maximum amount of memory that write buffers may take, as a fraction of the total shared memory. This option only has an effect when 'state.backend.forst.memory.managed' or 'state.backend.forst.memory.fixed-per-slot' are configured. |
state.backend.forst.options-factory |
(none) |
String |
The options factory class for users to add customized options in DBOptions and ColumnFamilyOptions for ForSt. If set, the ForSt state backend will load the class and apply configs to DBOptions and ColumnFamilyOptions after loading ones from 'ForStConfigurableOptions' and pre-defined options. |
state.backend.forst.primary-dir |
"checkpoint-dir" |
String |
The primary directory where ForSt puts its SST files. By default, it will be the same as the checkpoint directory. Recognized shortcut name is 'checkpoint-dir', which means that ForSt shares the directory with checkpoint, and 'local-dir', which means that ForSt will use the local directory of TaskManager. |
state.backend.forst.timer-service.cache-size |
128 |
Integer |
The cache size per keyGroup of ForSt timer service factory. This option only has an effect when 'state.backend.forst.timer-service.factory' is configured to 'ForStDB'. Increasing this value can improve the performance of ForSt timer service, but consumes more heap memory at the same time. |
state.backend.forst.timer-service.factory |
ForStDB |
Enum |
This determines the factory for timer service state implementation.
Possible values:- "HEAP": Heap-based
- "ForStDB": Implementation based on RocksDB
|