操作系统诊断工具truss, pstack, and pmap

操作系统诊断工具truss, pstack, and pmap
truss
Truss is an UNIX utility to trace Unix System Calls in Solaris platform. Truss utility is very useful to
understand complex problems at OS level. As Truss utility generates enormous amount of data,
Oracle Database Instrumentation is always a first step to troubleshoot a problem. If the problem
cannot be distilled by Oracle Database instrumentation, then the use of OS tools such as truss,
pstack etc are required.

To trace a process and print minimal information:
truss –p <pid> Example: truss –p 23898
To trace a process, follow its children and print minimal information:
truss –f –p <pid> Example: truss –f –p 23898
To trace a process, print timestamp and print minimal information:
truss –d –p <pid> Example: truss –d –p 23898
To trace a process, send output to a file and print minimal information:
truss –o /tmp/truss.out –p <pid>
Example: truss –o /tmp/truss.out –d –p 23898

Utility in UNIX platform
Platform Name
Solaris  truss
HP-UX    TUSC( truss)
Linux    strace
AIX      truss

Truss of DBWR with –d –D flag
$ truss -d -D -p 1473 |more
Truss of DBWR with –d –D flag
$ truss -d -E -p 19001 |more
Truss of DB startup with –f flag
$ truss -d -E –f -p 2522 |more
在Linux下面
$strace -ttT -p 5164

AIX & HP
HP-UX operating system provides TUSC a tool similar to truss utility. But, truss is soft linked to tusc
utility and supports many options as in truss utility in Solaris. Essentially, discussions for Solaris,
applies to HP-UX platform too.
In AIX, truss is available too. But, truss in AIX does not support -E flag. There is no easy way to
measure amount of time elapsed in the system call itself using truss utility(at least, not that I know of).
pmap
pmap command can be used to understand virtual memory mapping of an UNIX process, memory
usage, and attributes of memory area etc.
In AIX, this tool is named as procmap
In Linux, & HP-UX, pmap is available
pmap –xs output
$ pmap -x 2540 |more

pmap –xs output
Pmap –xs <pid>
pmap <pid>
$ pmap -x 2540 |more
2540: ora_pmon_solrac2
Address Kbytes RSS Anon Locked Mode Mapped File
0000000000400000 232736 37864 - - r-x-- oracle
000000000E757000 1424 476 144 - rw--- oracle
000000000E8BB000 156 32 32 - rw--- oracle
000000000E8E2000 1972 1132 1124 - rw--- [ heap ]
0000000060000000 526336 321508 - - rwxs- [ dism
shmid=0x3b ]
FFFFFD7FFCAA0000 64 - - - rwx-- [ anon ]
FFFFFD7FFCABE000 72 8 8 - rw--- [ anon ]
FFFFFD7FFCAD0000 64 12 12 - rw--- [ anon ]
FFFFFD7FFCAE0000 64 20 20 - rw--- [ anon ]
..
---------------- ---------- ---------- ---------- ----------
total Kb 780044 368316 2072 -
Pmap prints a Nice memory map
of the Process. Various heaps and
Stacks are printed here

pmap
#! /bin/ksh
pid=$1
(( cnt=1000 ))
while [[ $cnt -gt 0 ]];
do
date
pmap -x $pid
pstack $pid
echo $cnt
(( cnt=cnt-1 ))
sleep 10
done
---A small shell script, to dump Memory map and stack of a process, in a loop,every 10 seconds

pstack
Utility pstack can print the current execution stack of a process. Oracle Database
processes are instrumented to measure performance and most performance issues can be
resolved using that instrumentation. Database processes are usually in one of these three
states, either executing a piece of code, waiting for an event such as I/O, or waiting in
CPU scheduling queue to be scheduled.
To measure performance of a program, optimal task is to alter the session to enable sql
trace and execute the program. But, that is not always possible in a production
environment. So, if a program is running longer in production, you can review the ASH
data to measure the waits. But, if the program is not suffering from waits and spending
time executing in CPU, how do you measure the performance of a program?
Utility pstack comes handy in these situations. Essentially, current execution stack of a
process can be printed using pstack utility. Using pstack in a loop, you can generate many
samples of execution stack of a process; with some aggregation, you can understand the
performance of a process better.
In AIX, this tool is named as procstack
pstack output
$pstack 2544

Pfiles:
? pfiles can be used to associate this file ids with file names.
? Pfiles lists the files currently opened by a process. In few
unix platform, this can be achieved by lsof command
also.

Many tools available, aka proc tools
pflags, pcred, pldd, psig, pstack, pfiles, pwdx,
pstop, prun, pwait, ptree, ptime

Oradebug utility can be used to print the call stack too. In Listing 1-14, call stack of the
process is printed in the current process, using setmypid command.
Listing 1-14 oradebug
SQL> oradebug setmypid
Statement processed.
SQL> oradebug short_stack
ksedsts()+1123<-ksdxfstk()+33<-ksdxen_int()+5127<-
ksdxen()+14<-opiodr()+1075<-ttcpip()+1433<-opitsk()+1536<-
opiino()+1653<-opiodr()+1075<-opidrv()+814<-sou2o()+87<-
opimai_real()+537<-ssthrdmain()+334<-main()+203<-
_start()+108
SQL> oradebug short_stack
qksedsts()+1123<-ksdxfstk()+33<-ksdxen_int()+5127<-
ksdxen()+14<-opiodr()+1075<-ttcpip()+1433<-opitsk()+1536<-
opiino()+1653<-opiodr()+1075<-opidrv()+814<-sou2o()+87<-
opimai_real()+537<-ssthrdmain()+334<-main()+203<-
_start()+108