Usage:
lucid perf
lucid perf --explain [--CATEGORY1 [--CATEGORY2]]
lucid perf --filter GROUP1,GROUP2,...
lucid perf --CATEGORY1 [--CATEGORY2]] [--extended|all] [--seconds seconds]
lucid perf --CATEGORY1 [--filter GROUP1,GROUP2,...] [--seconds seconds]
lucid perf --CATEGORY1 [METRIC1,METRIC2,...] [--seconds seconds]

Periodically output performance metrics.

Options:

--explain List and explain all available perf metrics.
--seconds seconds Time interval in seconds at which to print the perf metrics. (default: 2)
--no-timestamp Omit timestamps from the performance metrics output
--default Show default list of metrics for each category.
--extended Show extended list of metrics for each category.
--all Show full list of all metrics for each selected category.
--filter groups Show only metrics for the specified (comma separated) groups (see option descriptions for the available groups).
--fail-on-error Fail on perf retrieval errors
--time [metrics] Category `time`. Groups: time. Metrics: currentDateAndTimeUTC.
--fs [metrics] Category `fs`. Groups: concurrency,iops,latency,notifications,size,throughput,total. Metrics: readBytes,writeBytes,reads,writes,opens,creates,dels,infos,moves,locks,other,ops,readFails,writeFails,openFails,createFails,delFails,infoFails,moveFails,lockFails,otherFails,opFails,readTime,writeTime,openTime,createTime,delTime,infoTime,moveTime,lockTime,otherTime,opTime,timedoutOps,readSize,writeSize,pendReads,pendWrites,pendOpens,pendCreates,pendDels,pendInfos,pendMoves,pendLocks,pendOther,pendOps,totalRead,totalWritten,notifActive,notifPending,notifProcessed,notifMetaPend,notifPathPend,notifPathDelPend,notifAttrPend,notifMeta,notifPath,notifPathDel,notifAttr.
--encryptor [metrics] Category `encryptor`. Groups: concurrency,throughput,total. Metrics: encryptBytes,decryptBytes,prlEncrypts,prlDecrypts,decryptFails.
--compressor [metrics] Category `compressor`. Groups: concurrency,ratio,throughput,time,total. Metrics: currentDateAndTimeUTC,comprRatio,comprReadBytes,uncomprReadBytes,comprWriteBytes,uncomprWritesBytes,prlDecompresses,prlCompresses,totalComprReadBytes,totalUncomprReadBytes,totalComprWriteBytes,totalUncomprWriteBytes.
--prefetcher [metrics] Category `prefetcher`. Groups: concurrency,efficiency,iops,throughput. Metrics: %covered,%accurate,prefetchBytes,totalPrefetchBytes,prodBytes,totalProdBytes,prefetches,prodPrefetches,pendPrefetches,streams.
--cache [metrics] Category `cache`. Groups: concurrency,efficiency,iops,latency,pinning,size,throughput,utilization. Metrics: %hits,readBytes,hitBytes,missBytes,prefetchBytes,writeBytes,bckWriteBytes,reads,hits,misses,prefetches,writes,bckWrites,maxBckWrites,failedBckWrites,invalidateReqs,invalidatedRanges,readTime,hitTime,missTime,prefetchTime,writeTime,bckWriteTime,readSize,writeSize,pendReads,pendPrefetches,pendWrites,pendBckWrites,pendStoreReads,pendStoreWrites,bckWritesLimit,pendInvalidates,dirtyBytes,cachedBytes,%,persistedInvalidates,pagesPinned,pagesUnpinned,failedPagePins.
--logstore [metrics] Category `logstore`. Groups: concurrency,efficiency,iops,latency,throughput. Metrics: %hits,dataReadBytes,layoutReadBytes,totalWriteBytes,layoutWriteBytes,writes,dataReads,layoutHits,layoutMisses,schedWrites,activeWrites,pendComplWrites,getDataTime,WriteTime,writeActiveTime,writePendingTime.
--objectstore [metrics] Category `objectstore`. Groups: concurrency,iops,latency,throughput,total. Metrics: getBytes,putBytes,listEntries,schedGets,schedPuts,schedDels,schedLists,activeGets,activePuts,activeDels,activeLists,totalGetBytes,totalPutBytes,totalListEntries,gets,puts,dels,lists,getFails,putFails,delFails,listFails,getCanceled,putCanceled,getTime,putTime,delTime,listTime.
--keystore [metrics] Category `keystore`. Groups: iops,latency. Metrics: commits,aborts,puts,dels,getiters,finds,commitTime.
--versionedKeystore [metrics] Category `versionedKeystore`. Groups: concurrency,iops,latency. Metrics: commits,flushes,getiters,commitTime,getIterTime,flushTime,pendFlush.
--synchronizer [metrics] Category `synchronizer`. Groups: iops,latency,throughput. Metrics: localFetches,remoteFetches,localApplies,remoteApplies,localFetchedKeys,remoteFetchedKeys,localAppliedKeys,remoteAppliedKeys,localFetchTime,remoteFetchTime,localApplyTime,remoteApplyTime.
--preloader [metrics] Category `preloader`. Groups: concurrency,iops,throughput,total. Metrics: bytesFetched,fetches,files,dirs,totalBytes,totalFiles,totalDirs,pendFetches,activePreloaders.
--metastore [metrics] Category `metastore`. Groups: concurrency,efficiency,iops,latency. Metrics: fetches/misses,prefetches,fetchedRanges,prefetchedRanges,fetchedKeys,prefetchedKeys,fetchedRangesTime,prefetchedRangesTime,hits,pushes,pulls,pushedKeys,pulledKeys,pushTime,pullTime,localUpdates,remoteUpdates,localUpdateKeys,remoteUpdateKeys,evictedKeys,evictedRanges,dbSize,cachedNotifies,allNotifies,GCCollectedKeys,GCProcessedKeys,GCProcessedKeys,commits,flushes,getIters,commitTime,getIterTime,flushTime,pendFlush.

Usage:

lucid perf --option --categorys [metric1],[metric2],[metric...]

lucid perf --explain

--- time ---
currentDateAndTimeUTC: Current UTC date&time (group: time)

--- fs ---
readBytes: Read throughput (group: throughput)
writeBytes: Write throughput (group: throughput)
reads: Read requests/s (group: iops)
writes: Write requests/s (group: iops)
opens: Open requests/s (group: iops)
creates: Create requests/s (group: iops)
dels: Delete requests/s (group: iops)
infos: Get info requests/s (group: iops)
moves: Move requests/s (group: iops)
locks: Lock requests/s (group: iops)
other: Other requests/s (group: iops)
ops: Total requests/s (group: iops)
readFails: Read failures/s (group: iops)
writeFails: Write failures/s (group: iops)
openFails: Open failures/s (group: iops)
createFails: Create failures/s (group: iops)
delFails: Delete failures/s (group: iops)
infoFails: Get info failures/s (group: iops)
moveFails: Move failures/s (group: iops)
lockFails: Lock failures/s (group: iops)
otherFails: Other request failures/s (group: iops)
opFails: Total request failures/s (group: iops)
readTime: Average read time (group: latency)
writeTime: Average write time (group: latency)
openTime: Average open time (group: latency)
createTime: Average create time (group: latency)
delTime: Average delete time (group: latency)
infoTime: Average get info time (group: latency)
moveTime: Average move time (group: latency)
lockTime: Average lock time (group: latency)
otherTime: Average other time (group: latency)
opTime: Average request time (group: latency)
timedoutOps: Total timedout ops (group: latency)
readSize: Average read size (group: size)
writeSize: Average write size (group: size)
pendReads: Outstanding read requests (group: concurrency)
pendWrites: Outstanding write requests (group: concurrency)
pendOpens: Outstanding open requests (group: concurrency)
pendCreates: Outstanding create requests (group: concurrency)
pendDels: Outstanding delete requests (group: concurrency)
pendInfos: Outstanding get info requests (group: concurrency)
pendMoves: Outstanding move requests (group: concurrency)
pendLocks: Outstanding lock requests (group: concurrency)
pendOther: Outstanding other requests (group: concurrency)
pendOps: Total outstanding requests (group: concurrency)
totalRead: Total bytes read (group: total)
totalWritten: Total bytes written (group: total)
notifActive: Active notifications (group: notifications)
notifPending: Pending notifications (group: notifications)
notifProcessed: Processed notifications/s (group: notifications)
notifMetaPend: Metadata notifications pending (group: notifications)
notifPathPend: Path notifications pending (group: notifications)
notifPathDelPend: Path delete notifications pending (group: notifications)
notifAttrPend: Attribute notifications pending (group: notifications)
notifMeta: Metadata notifications/s (group: notifications)
notifPath: Path notifications/s (group: notifications)
notifPathDel: Path delete notifications/s (group: notifications)
notifAttr: Attribute notifications/s (group: notifications)

--- encryptor ---
encryptBytes: Encrypt throughput (group: throughput)
decryptBytes: Decrypt throughput (group: throughput)
prlEncrypts: Concurrent encrypt operations (group: concurrency)
prlDecrypts: Concurrent decrypt operations (group: concurrency)
decryptFails: Total decrypt failures (group: total)

--- compressor ---
currentDateAndTimeUTC: Current UTC date&time (group: time)
comprRatio: Compression ratio uncompressed/compressed (group: ratio)
comprReadBytes: Compressed read throughput (group: throughput)
uncomprReadBytes: Uncompressed read throughput (group: throughput)
comprWriteBytes: Compressed write throughput (group: throughput)
uncomprWritesBytes: Uncompressed write throughput (group: throughput)
prlDecompresses: Concurrent decompress operations (group: concurrency)
prlCompresses: Concurrent compress operations (group: concurrency)
totalComprReadBytes: Total compressed bytes read (group: total)
totalUncomprReadBytes: Total uncompressed bytes read (group: total)
totalComprWriteBytes: Total compressed bytes written (group: total)
totalUncomprWriteBytes: Total uncompressed bytes written (group: total)

--- prefetcher ---
%covered: Coverage % (group: efficiency)
%accurate: Accuracy % (group: efficiency)
prefetchBytes: Prefetch read throughput (group: throughput)
totalPrefetchBytes: Total prefetch read bytes (group: throughput)
prodBytes: Productive prefetch read throughput (group: throughput)
totalProdBytes: Total productive prefetch read bytes (group: throughput)
prefetches: Prefetch reads/s (group: iops)
prodPrefetches: Productive prefetch reads/s (group: iops)
pendPrefetches: Outstanding prefetch reads (group: concurrency)
streams: Prefetch streams (group: concurrency)

--- cache ---
%hits: Read hit % (group: efficiency)
readBytes: Total read throughput (group: throughput)
hitBytes: Cached(hit) read throughput (group: throughput)
missBytes: Backend(miss) read throughput (group: throughput)
prefetchBytes: Prefetch read throughput (group: throughput)
writeBytes: Total write throughput (group: throughput)
bckWriteBytes: Backend write throughput (group: throughput)
reads: Total reads/s (group: iops)
hits: Cached(hit) reads/s (group: iops)
misses: Backend(miss) reads/s (group: iops)
prefetches: Prefetches/s (group: iops)
writes: Total writes/s (group: iops)
bckWrites: Backend writes/s (group: iops)
maxBckWrites: Maximum backend writes (group: iops)
failedBckWrites: Failed backend writes/s (group: iops)
invalidateReqs: Invalidate requests/s (group: iops)
invalidatedRanges: Invalidated ranges/s (group: iops)
readTime: Average read time (group: latency)
hitTime: Average cached(hit) read time (group: latency)
missTime: Average backend(miss) read time (group: latency)
prefetchTime: Average prefetch time (group: latency)
writeTime: Average write time (group: latency)
bckWriteTime: Average backend write time (group: latency)
readSize: Average read size (group: size)
writeSize: Average write size (group: size)
pendReads: Outstanding reads (group: concurrency)
pendPrefetches: Outstanding prefetches (group: concurrency)
pendWrites: Outstanding writes (group: concurrency)
pendBckWrites: Outstanding backend writes (group: concurrency)
pendStoreReads: Outstanding cache store reads (group: concurrency)
pendStoreWrites: Outstanding cache store writes (group: concurrency)
bckWritesLimit: Backend writes limit (group: concurrency)
pendInvalidates: Outstanding invalidations (group: concurrency)
dirtyBytes: Dirty data (group: utilization)
cachedBytes: Cached data (group: utilization)
%: Dirty % (group: utilization)
persistedInvalidates: Persisted invalidations (group: utilization)
pagesPinned: Page pins (group: pinning)
pagesUnpinned: Page unpins (group: pinning)
failedPagePins: Page failed pins (group: pinning)

--- logstore ---
%hits: Layout read hit % (group: efficiency)
dataReadBytes: Total read throughput (group: throughput)
layoutReadBytes: Layout read throughput (group: throughput)
totalWriteBytes: Write throughput (group: throughput)
layoutWriteBytes: Layout write throughput (group: throughput)
writes: Total writes/s (group: iops)
dataReads: Total reads/s (group: iops)
layoutHits: Layout read hits/s (group: iops)
layoutMisses: Layout read misses/s (group: iops)
schedWrites: Scheduled writes (group: concurrency)
activeWrites: Active writes (group: concurrency)
pendComplWrites: Pending completion writes (group: concurrency)
getDataTime: Average read time (group: latency)
WriteTime: Average write time (group: latency)
writeActiveTime: Average write active time (group: latency)
writePendingTime: Average write pending completion time (group: latency)

--- objectstore ---
getBytes: GET throughput (group: throughput)
putBytes: PUT throughput (group: throughput)
listEntries: LIST entries/ (group: throughput)
schedGets: Scheduled GET requests (group: concurrency)
schedPuts: Scheduled PUT requests (group: concurrency)
schedDels: Scheduled DELETE requests (group: concurrency)
schedLists: Scheduled LIST requests (group: concurrency)
activeGets: Active GET requests (group: concurrency)
activePuts: Active PUT requests (group: concurrency)
activeDels: Active DELETE requests (group: concurrency)
activeLists: Active LIST requests (group: concurrency)
totalGetBytes: Total GET bytes (group: total)
totalPutBytes: Total PUT bytes (group: total)
totalListEntries: Total LIST entries (group: total)
gets: GET requests/s (group: iops)
puts: PUT requests/s (group: iops)
dels: DELETE requests/s (group: iops)
lists: LIST requests/s (group: iops)
getFails: GET failures/s (group: iops)
putFails: PUT failures/s (group: iops)
delFails: DELETE failures/s (group: iops)
listFails: LIST failures/s (group: iops)
getCanceled: GET cancelations/s (group: iops)
putCanceled: PUT cancelations/s (group: iops)
getTime: Average GET request time (group: latency)
putTime: Average PUT request time (group: latency)
delTime: Average DELETE request time (group: latency)
listTime: Average LIST request time (group: latency)

--- keystore ---
commits: Commits/s (group: iops)
aborts: Aborted transactions/s (group: iops)
puts: Puts/s (group: iops)
dels: Deletes/s (group: iops)
getiters: GetIterator calls/s (group: iops)
finds: Get/Exists calls/s (group: iops)
commitTime: Average commit time (group: latency)

--- versionedKeystore ---
commits: Commits/s (group: iops)
flushes: Flushes/s (group: iops)
getiters: GetIterator calls/s (group: iops)
commitTime: Average commit time (group: latency)
getIterTime: Average get iterator time (group: latency)
flushTime: Average flush time (group: latency)
pendFlush: Outstanding flushes (group: concurrency)

--- synchronizer ---
localFetches: Local FetchUpdates/s (group: iops)
remoteFetches: Remote FetchUpdates/s (group: iops)
localApplies: Local ApplyUpdates/s (group: iops)
remoteApplies: Remote ApplyUpdates/s (group: iops)
localFetchedKeys: Local keys fetched/s (group: throughput)
remoteFetchedKeys: Remote keys fetched/s (group: throughput)
localAppliedKeys: Local keys applied/s (group: throughput)
remoteAppliedKeys: Remote keys applied/s (group: throughput)
localFetchTime: Average local FetchUpdates time (group: latency)
remoteFetchTime: Average remote FetchUpdates time (group: latency)
localApplyTime: Average local ApplyUpdates time (group: latency)
remoteApplyTime: Average remote ApplyUpdates time (group: latency)

--- preloader ---
bytesFetched: Preload throughput (group: throughput)
fetches: Fetch reads/s (group: iops)
files: Files preloaded/s (group: iops)
dirs: Dirs scanned/s (group: iops)
totalBytes: Total preload bytes (group: total)
totalFiles: Total files preloaded (group: total)
totalDirs: Total directories scanned (group: total)
pendFetches: Outstanding preload reads (group: concurrency)
activePreloaders: Active preloaders (group: concurrency)

--- metastore ---
fetches/misses: Fetches/s (group: iops)
prefetches: Prefetches/s (group: iops)
fetchedRanges: Fetch ranges/s (group: iops)
prefetchedRanges: Prefetch ranges/s (group: iops)
fetchedKeys: Fetched keys/s (group: iops)
prefetchedKeys: Prefetched keys/s (group: iops)
fetchedRangesTime: Average fetched ranges time (group: latency)
prefetchedRangesTime: Average prefetched ranges time (group: latency)
hits: Cache hits/s (group: iops)
pushes: Pushes/s (group: iops)
pulls: Pulls/s (group: iops)
pushedKeys: Pushed keys/s (group: iops)
pulledKeys: Pulled keys/s (group: iops)
pushTime: Average push time (group: latency)
pullTime: Average pull time (group: latency)
localUpdates: Local updates/s (group: iops)
remoteUpdates: Remote updates/s (group: iops)
localUpdateKeys: Local update keys/s (group: iops)
remoteUpdateKeys: Remote update keys/s (group: iops)
evictedKeys: Evicted keys/s (group: iops)
evictedRanges: Evicted ranges/s (group: iops)
dbSize: Current database size (group: efficiency)
cachedNotifies: Notifications/s (group: iops)
allNotifies: Notifications/s (group: iops)
GCCollectedKeys: GC collected keys/s (group: iops)
GCProcessedKeys: GC processed keys/s (group: iops)
GCProcessedKeys: GC commits (group: iops)
commits: Commits/s (group: iops)
flushes: Flushes/s (group: iops)
getIters: GetIterator calls/s (group: iops)
commitTime: Average commit time (group: latency)
getIterTime: Average get iterator time (group: latency)
flushTime: Average flush time (group: latency)
pendFlush: Outstanding flushes (group: concurrency)

All metrics:

lucid perf --all

Remaining uploads:

lucid perf --cache dirtyBytes

Cache data:

lucid perf --cache cachedBytes

Cache group filters iops and throughput

lucid perf --cache --filter iops,throughput

Filesystem reads,writes with cached data in 1 second intervals:

lucid perf --seconds 1 --fs reads,writes --cache dirtyBytes,cachedBytes

--seconds 0 provides a single output