Some time you want to get the quick information about Cell like offloading, Storage index etc. Exadata cell has a small tool named as cellsrvstat which provides the comprehensive information like below
I/O related statistics
Memory related statistics
Network related statistics
smart I/O related statistics
Memory related statistics
Network related statistics
smart I/O related statistics
[root@exacell1 ~]# cellsrvstat
===Current Time=== Tue Oct 29 22:36:42 2013
== Input/Output related stats ==
Number of hard disk block IO read requests 0 51684
Number of hard disk block IO write requests 0 40896
Hard disk block IO reads (KB) 0 1251509
Hard disk block IO writes (KB) 0 445961
Number of flash disk block IO read requests 0 11400
Number of flash disk block IO write requests 0 23568
Flash disk block IO reads (KB) 0 116218
Flash disk block IO writes (KB) 0 494972
Number of disk IO errors 0 0
Number of reads from flash cache 0 11340
Number of writes to flash cache 0 23235
Flash cache reads (KB) 0 116216
Flash cache writes (KB) 0 0
Number of flash cache IO errors 0 0
Size of eviction from flash cache (KB) 0 0
Number of outstanding large flash IOs 0 0
Number of latency threshold warnings during job 0 19
Number of latency threshold warnings by checker 0 0
Number of latency threshold warnings for smart IO 0 0
Number of latency threshold warnings for redo log writes 0 0
Current read block IO to be issued (KB) 0 0
Total read block IO to be issued (KB) 0 975737
Current write block IO to be issued (KB) 0 0
Total write block IO to be issued (KB) 0 435721
Current read blocks in IO (KB) 0 0
Total read block IO issued (KB) 0 975737
Current write blocks in IO (KB) 0 0
Total write block IO issued (KB) 0 435721
Current read block IO in network send (KB) 0 0
Total read block IO in network send (KB) 0 975737
Current write block IO in network send (KB) 0 0
Total write block IO in network send (KB) 0 435721
Current block IO being populated in flash (KB) 0 0
Total block IO KB populated in flash (KB) 0 373596
== Memory related stats ==
SGA heap used - kgh statistics (KB) 0 577025
SGA heap free - cellsrv statistics (KB) 0 12800
OS memory allocated to SGA (KB) 0 589827
SGA heap used - cellsrv statistics - KB 0 577027
OS memory allocated to PGA (KB) 0 8831
PGA heap used - cellsrv statistics (KB) 0 425
OS memory allocated to cellsrv (KB) 0 3766329
Top 5 SGA consumers (KB)
SUBHEAP Networ 0 294971
FlashCacheCtx 0 99618
Thread IO Lat Stats 0 56252
RemoteSendPort Fixed Size 0 33888
RemoteOpenInfo Fixed Size 0 22504
Top 5 SGA subheap consumers (KB)
Network mem 0 294930
Network heap chunk 0 6408
kutyxtte_deserialize_internal 0 1
kutyxtte_deserialize_internal 0 1
kutyxtte_deserialize_internal 0 1
Number of allocation failures in 512 bytes pool 0 0
Number of allocation failures in 2KB pool 0 0
Number of allocation failures in 4KB pool 0 0
Number of allocation failures in 8KB pool 0 0
Number of allocation failures in 16KB pool 0 0
Number of allocation failures in 32KB pool 0 0
Number of allocation failures in 64KB pool 0 0
Number of allocation failures in 1MB pool 0 0
Allocation hwm in 512 bytes pool 0 612
Allocation hwm in 2KB pool 0 603
Allocation hwm in 4KB pool 0 612
Allocation hwm in 8KB pool 0 1161
Allocation hwm in 16KB pool 0 610
Allocation hwm in 32KB pool 0 609
Allocation hwm in 64KB pool 0 683
Allocation hwm in 1MB pool 0 60
Number of low memory threshold failures 0 0
Number of no memory threshold failures 0 0
Dynamic buffer allocation requests 0 0
Dynamic buffer allocation failures 0 0
Dynamic buffer allocation failures due to low mem 0 0
Dynamic buffer allocated size (KB) 0 0
Dynamic buffer allocation hwm (KB) 0 0
== Execution related stats ==
Incarnation number 0 21
Number of module version failures 0 0
Number of threads working 0 1
Number of threads waiting for network 0 18
Number of threads waiting for resource 0 0
Number of threads waiting for a mutex 0 0
Number of Jobs executed for each job type
CacheGet 0 63024
CachePut 0 40885
CloseDisk 0 693
OpenDisk 0 1267
ProcessIoctl 0 23471
PredicateDiskRead 0 0
PredicateDiskWrite 0 1
PredicateFilter 0 1
PredicateCacheGet 0 3
PredicateCachePut 0 1
FlashCacheMetadataWrite 0 0
RemoteListenerJob 0 0
FlashCacheResilveringTableUpdate 0 0
CellDiskMetadataPrepare 0 0
SQL ids consuming the most CPU
other 0000000000000 11
END SQL ids consuming the most CPU
== Network related stats ==
Total bytes received from the network 0 471836816
Total bytes transmitted to the network 0 1006633192
Total bytes retransmitted to the network 0 64488
Number of active sendports 0 11
Hwm of active sendports 0 32
Number of active remote open infos 0 40
HWM of remote open infos 0 140
== SmartIO related stats ==
Number of active smart IO sessions 0 0
High water mark of smart IO sessions 0 1
Number of completed smart IO sessions 0 1
Smart IO offload efficiency (percentage) 0 0
Size of IO avoided due to storage index (KB) 0 0
Current smart IO to be issued (KB) 0 0
Total smart IO to be issued (KB) 0 10240
Current smart IO in IO (KB) 0 0
Total smart IO in IO (KB) 0 10240
Current smart IO being cached in flash (KB) 0 0
Total smart IO being cached in flash (KB) 0 0
Current smart IO with IO completed (KB) 0 0
Total smart IO with IO completed (KB) 0 10240
Current smart IO being filtered (KB) 0 0
Total smart IO being filtered (KB) 0 10240
Current smart IO filtering completed (KB) 0 0
Total smart IO filtering completed (KB) 0 10240
Current smart IO filtered size (KB) 0 0
Total smart IO filtered (KB) 0 0
Total cpu passthru output IO size (KB) 0 0
Total passthru output IO size (KB) 0 0
Current smart IO with results in send (KB) 0 0
Total smart IO with results in send (KB) 0 10240
Current smart IO filtered in send (KB) 0 0
Total smart IO filtered in send (KB) 0 0
Total smart IO read from flash (KB) 0 0
Total smart IO initiated flash population (KB) 0 0
Total smart IO read from hard disk (KB) 0 0
Total smart IO writes (fcre) to hard disk (KB) 0 10240
Number of smart IO requests < 512KB 0 0
Number of smart IO requests >= 512KB and < 1MB 0 0
Number of smart IO requests >= 1MB and < 2MB 0 0
Number of smart IO requests >= 2MB and < 4MB 0 2
Number of smart IO requests >= 4MB and < 8MB 0 1
Number of smart IO requests >= 8MB 0 0
Number of times smart IO buffer reserve failures 0 0
Number of times smart IO request misses 0 0
Number of times IO for smart IO not allowed to be issued 0 0
Number of times smart IO prefetch limit was reached 0 0
Number of times smart scan used unoptimized mode 0 0
Number of times smart fcre used unoptimized mode 0 0
Number of times smart backup used unoptimized mode 0 0
[root@exacell1 ~]#
You can use another option -list to query what metrics you want to observe
[root@exacell1 ~]# cellsrvstat -list
Statistic Groups:
[root@exacell1 ~]# cellsrvstat -list
Statistic Groups:
io Input/Output related stats
mem Memory related stats
exec Execution related stats
net Network related stats
smartio SmartIO related stats
Statistics:
[ * - Absolute values. Indicates no delta computation in tabular format]
io_nbiorr_hdd Number of hard disk block IO read requests
io_nbiowr_hdd Number of hard disk block IO write requests
io_nbiorb_hdd Hard disk block IO reads (KB)
io_nbiowb_hdd Hard disk block IO writes (KB)
io_nbiorr_flash Number of flash disk block IO read requests
io_nbiowr_flash Number of flash disk block IO write requests
io_nbiorb_flash Flash disk block IO reads (KB)
io_nbiowb_flash Flash disk block IO writes (KB)
io_ndioerr Number of disk IO errors
io_nrfc Number of reads from flash cache
io_nwfc Number of writes to flash cache
io_fcrb Flash cache reads (KB)
io_fcwb Flash cache writes (KB)
io_nfioerr Number of flash cache IO errors
io_nbpfce Size of eviction from flash cache (KB)
io_nolfio Number of outstanding large flash IOs
io_ltow Number of latency threshold warnings during job
io_ltcw Number of latency threshold warnings by checker
io_ltsiow Number of latency threshold warnings for smart IO
io_ltrlw Number of latency threshold warnings for redo log writes
io_bcrti Current read block IO to be issued (KB) *
io_btrti Total read block IO to be issued (KB)
io_bcwti Current write block IO to be issued (KB) *
io_btwti Total write block IO to be issued (KB)
io_bcrii Current read blocks in IO (KB) *
io_btrii Total read block IO issued (KB)
io_bcwii Current write blocks in IO (KB) *
io_btwii Total write block IO issued (KB)
io_bcrsi Current read block IO in network send (KB) *
io_btrsi Total read block IO in network send (KB)
io_bcwsi Current write block IO in network send (KB) *
io_btwsi Total write block IO in network send (KB)
io_bcfp Current block IO being populated in flash (KB) *
io_btfp Total block IO KB populated in flash (KB)
mem_sgahu SGA heap used - kgh statistics (KB)
mem_sgahf SGA heap free - cellsrv statistics (KB)
mem_sgaos OS memory allocated to SGA (KB)
mem_sgahuc SGA heap used - cellsrv statistics - KB
mem_pgaos OS memory allocated to PGA (KB)
mem_pgahuc PGA heap used - cellsrv statistics (KB)
mem_allos OS memory allocated to cellsrv (KB)
mem_sgatop Top 5 SGA consumers (KB) *
mem_sgasubtop Top 5 SGA subheap consumers (KB) *
mem_halfkaf Number of allocation failures in 512 bytes pool
mem_2kaf Number of allocation failures in 2KB pool
mem_4kaf Number of allocation failures in 4KB pool
mem_8kaf Number of allocation failures in 8KB pool
mem_16kaf Number of allocation failures in 16KB pool
mem_32kaf Number of allocation failures in 32KB pool
mem_64kaf Number of allocation failures in 64KB pool
mem_1maf Number of allocation failures in 1MB pool
mem_halfkhwm Allocation hwm in 512 bytes pool
mem_2khwm Allocation hwm in 2KB pool
mem_4khwm Allocation hwm in 4KB pool
mem_8khwm Allocation hwm in 8KB pool
mem_16khwm Allocation hwm in 16KB pool
mem_32khwm Allocation hwm in 32KB pool
mem_64khwm Allocation hwm in 64KB pool
mem_1mhwm Allocation hwm in 1MB pool
mem_lmtf Number of low memory threshold failures
mem_nmtf Number of no memory threshold failures
mem_dynar Dynamic buffer allocation requests
mem_dynaf Dynamic buffer allocation failures
mem_dynafl Dynamic buffer allocation failures due to low mem
mem_dynam Dynamic buffer allocated size (KB)
mem_dynamh Dynamic buffer allocation hwm (KB)
exec_incno Incarnation number *
exec_versf Number of module version failures *
exec_ntwork Number of threads working *
exec_ntnetwait Number of threads waiting for network *
exec_ntreswait Number of threads waiting for resource *
exec_ntmutexwait Number of threads waiting for a mutex *
exec_njx Number of Jobs executed for each job type
exec_topcpusqlid SQL ids consuming the most CPU
net_rxb Total bytes received from the network
net_txb Total bytes transmitted to the network
net_rtxb Total bytes retransmitted to the network
net_sps Number of active sendports
net_sph Hwm of active sendports
net_rois Number of active remote open infos
net_roih HWM of remote open infos
sio_ns Number of active smart IO sessions *
sio_hs High water mark of smart IO sessions *
sio_ncs Number of completed smart IO sessions
sio_oe Smart IO offload efficiency (percentage) *
sio_sis Size of IO avoided due to storage index (KB)
sio_ctb Current smart IO to be issued (KB) *
sio_ttb Total smart IO to be issued (KB)
sio_cii Current smart IO in IO (KB) *
sio_tii Total smart IO in IO (KB)
sio_cfp Current smart IO being cached in flash (KB) *
sio_tfp Total smart IO being cached in flash (KB)
sio_cic Current smart IO with IO completed (KB) *
sio_tic Total smart IO with IO completed (KB)
sio_cif Current smart IO being filtered (KB) *
sio_tif Total smart IO being filtered (KB)
sio_cfc Current smart IO filtering completed (KB) *
sio_tfc Total smart IO filtering completed (KB)
sio_cfo Current smart IO filtered size (KB) *
sio_tfo Total smart IO filtered (KB)
sio_tcpo Total cpu passthru output IO size (KB)
sio_tpo Total passthru output IO size (KB)
sio_cis Current smart IO with results in send (KB) *
sio_tis Total smart IO with results in send (KB)
sio_ciso Current smart IO filtered in send (KB) *
sio_tiso Total smart IO filtered in send (KB)
sio_fcr Total smart IO read from flash (KB)
sio_fcw Total smart IO initiated flash population (KB)
sio_hdr Total smart IO read from hard disk (KB)
sio_hdw Total smart IO writes (fcre) to hard disk (KB)
sio_n512kb Number of smart IO requests < 512KB
sio_n1mb Number of smart IO requests >= 512KB and < 1MB
sio_n2mb Number of smart IO requests >= 1MB and < 2MB
sio_n4mb Number of smart IO requests >= 2MB and < 4MB
sio_n8mb Number of smart IO requests >= 4MB and < 8MB
sio_ngt8mb Number of smart IO requests >= 8MB
sio_nbrf Number of times smart IO buffer reserve failures
sio_nrm Number of times smart IO request misses
sio_ncio Number of times IO for smart IO not allowed to be issued
sio_nplr Number of times smart IO prefetch limit was reached
sio_nssuo Number of times smart scan used unoptimized mode
sio_nfcuo Number of times smart fcre used unoptimized mode
sio_nsbuo Number of times smart backup used unoptimized mode
Now you can get the information for any metric given in above output.
[root@exacell1 ~]# cellsrvstat -table -interval=1 -count=5 -stat=io_nbiorr_hdd,io_nbiowr_hdd
===Current Time=== io_nbiorr_hdd io_nbiowr_hdd
Tue Oct 29 22:56:43 2013 0 0
Tue Oct 29 22:56:44 2013 0 0
Tue Oct 29 22:56:45 2013 4 2
Tue Oct 29 22:56:46 2013 0 1
Tue Oct 29 22:56:47 2013 0 0
Omit -table for complete information. Use cellsrvstat -h for all options.
Note: OSwatcher on Exadata calls this utility also
[root@exacell1 ~]# ps -ef | grep osw
....
root 12914 23131 0 08:00 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_cellsrvstat.sh
root 31625 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_vmstat.sh
root 31626 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_mpstat.sh
root 31627 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_netstat.sh
root 31628 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_iostat.sh
root 31629 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_diskstats.sh
root 31633 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_top.sh
.....
You can view the scripts as below
[root@exacell1 ~]#cat /opt/oracle.oswatcher/osw/Exadata_cellsrvstat.sh
Related Posts:
Exadata: Monitoring Active Requests, Alerts and Wait Events
Exadata: Replacing damaged disk is really plugNplay activity
Exadata: What differentiates GI on Exadata with GI on non-Exadata?
Exadata: Understanding key OS Processes for a cell
Exadata: Health Checking Exadata
Exadata: Diagnostics using sundiag/deaddisk
Exadata: Knowing a bit Exadata administrative utilities
[root@exacell1 ~]# ps -ef | grep osw
....
root 12914 23131 0 08:00 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_cellsrvstat.sh
root 31625 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_vmstat.sh
root 31626 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_mpstat.sh
root 31627 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_netstat.sh
root 31628 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_iostat.sh
root 31629 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_diskstats.sh
root 31633 23131 0 04:02 ? 00:00:00 /bin/ksh ./oswsub.sh HighFreq ./Exadata_top.sh
.....
You can view the scripts as below
[root@exacell1 ~]#cat /opt/oracle.oswatcher/osw/Exadata_cellsrvstat.sh
Related Posts:
Exadata: Monitoring Active Requests, Alerts and Wait Events
Exadata: Replacing damaged disk is really plugNplay activity
Exadata: What differentiates GI on Exadata with GI on non-Exadata?
Exadata: Understanding key OS Processes for a cell
Exadata: Health Checking Exadata
Exadata: Diagnostics using sundiag/deaddisk
Exadata: Knowing a bit Exadata administrative utilities
No comments:
Post a Comment