Server : Apache System : Linux server2.corals.io 4.18.0-348.2.1.el8_5.x86_64 #1 SMP Mon Nov 15 09:17:08 EST 2021 x86_64 User : corals ( 1002) PHP Version : 7.4.33 Disable Function : exec,passthru,shell_exec,system Directory : /opt/rh/gcc-toolset-11/root/usr/share/systemtap/examples/process/ |
cycle_thief.stp - Track IRQ's and Other Processes Stealing Cycles from a Task
Description:
The cycle_thief.stp script instruments the scheduler and IRQ handler
to determine which processes and interrupts are competing with the
specified task for the cpu cycles. This script uses the '-c' or '-x'
options to focus on a specific task. The script output the number of
times the task migrates between processors, histograms showing the
length of time on and off processor, lists of processes running while
the task is off the processor, and the interrupts that occurred while
the task was running.
$ stap cycle_thief.stp
task 0 migrated: 1
task 0 on processor (us):
value |-------------------------------------------------- count
2 | 0
4 | 0
8 |@ 1
16 | 0
32 | 0
64 | 0
128 |@@@@@@@@ 8
256 |@ 1
512 | 0
1024 |@ 1
2048 |@@@@@@@@ 8
4096 |@@@@@@@@@@@ 11
8192 |@@ 2
16384 |@@ 2
32768 |@@@@@@@@@@@@ 12
65536 |@@@@@ 5
131072 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 30
262144 | 0
524288 | 0
task 0 off processor (us)
value |-------------------------------------------------- count
1 | 0
2 | 0
4 |@@@@ 4
8 |@@@@@@@@@@@@@@@@@@@@@@@@@@@ 27
16 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 34
32 |@@@@ 4
64 |@@@@@ 5
128 |@@@@@ 5
256 | 0
512 |@ 1
1024 | 0
2048 | 0
other pids taking processor from task 0
PID count command
25145 33 stapio
19189 17 kworker/0:3
7 13 rcu_sched
22717 9 kworker/1:1
380 8 jbd2/dm-0-8
9 8 rcuos/1
570 6 kworker/0:1H
8 3 rcuos/0
478 3 systemd-journal
24747 3 kworker/0:1
491 2 systemd-udevd
19120 2 sshd
25142 2 stapio
615 1 alsactl
625 1 irqbalance
25144 1 stapio
31816 1 kworker/u4:1
25141 1 stap
724 1 NetworkManager
25143 1 systemd-udevd
irq taking processor from task 0
irq count min(us) avg(us) max(us)
44 9 3 8 19
The output from running the script for ~10 seconds.
The histograms provide a visual of the time a process spent on, or off. The
sampled processes are listed below the histograms to give a sense of what
was running at the time.
As mentioned in the description, the script can be focused on a certain process
with -c/-x.
$ stap cycle_thief.stp -T 10
task 29018 migrated: 2
task 29018 on processor (us):
value |-------------------------------------------------- count
2 | 0
4 | 0
8 |@ 1
16 | 0
32 |@ 1
64 |@ 1
128 | 0
256 |@ 1
512 | 0
1024 | 0
task 29018 off processor (us)
value |-------------------------------------------------- count
2 | 0
4 | 0
8 |@ 1
16 |@ 1
32 | 0
64 | 0
~
2097152 | 0
4194304 | 0
8388608 |@ 1
16777216 | 0
33554432 | 0
other pids taking processor from task 29018
PID count command
0 41 swapper/0
19189 24 kworker/0:3
7 12 rcu_sched
8 8 rcuos/0
29002 5 kworker/0:0
380 4 jbd2/dm-0-8
570 3 kworker/0:1H
6 3 kworker/u4:0
491 2 systemd-udevd
29016 2 stapio
29017 1 systemd-udevd
478 1 systemd-journal
625 1 irqbalance
14 1 migration/1
irq taking processor from task 29018
irq count min(us) avg(us) max(us)