IRQ process using 60% CPU

Questions related to general functionality
hugleo
Posts: 59
Joined: 13 Apr 2016, 14:28

IRQ process using 60% CPU

Post by hugleo »

Is that normal for accel-ppp?
Attachments
zombie.png
zombie.png (53.59 KiB) Viewed 2640 times
Last edited by hugleo on 03 Feb 2017, 18:06, edited 1 time in total.
dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: Zombie process using 60% CPU

Post by dimka88 »

why zombie? =)

maybe interrupts network card.
Run command

Code: Select all

cat /proc/interrupts
hugleo
Posts: 59
Joined: 13 Apr 2016, 14:28

Re: Zombie process using 60% CPU

Post by hugleo »

Most of times I see ksoftirqd/0 with 90%. Maybe it should be balancend between all cores ksoftirqd/1, ksoftirqd/2...

Code: Select all


            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7       
   0:  338093413          0          0          0          0          0          0          0   IO-APIC-edge      timer
   1:          5          0          0          0          0          0          0          0   IO-APIC-edge      i8042
   8:          1          0          0          0          0          0          0          0   IO-APIC-edge      rtc0
   9:          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   acpi
  14:          0          0          0          0          0          0          0          0   IO-APIC-edge      ata_piix
  15:          0          0          0          0          0          0          0          0   IO-APIC-edge      ata_piix
  16:         43          0          0          0          0          0          0          0   IO-APIC-fasteoi   uhci_hcd:usb3
  18:          2          0          0          0          0          0          0          0   IO-APIC-fasteoi   ehci_hcd:usb1, uhci_hcd:usb6
  19:          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   uhci_hcd:usb5, ata_piix
  23:          0          0          0          0          0          0          0          0   IO-APIC-fasteoi   ehci_hcd:usb2, uhci_hcd:usb4
  66:     108272          0          0          0          0          0          0          0   PCI-MSI-edge      hpsa0
  67:        323     330783          0          0          0          0          0          0   PCI-MSI-edge      hpsa0
  68:        115          0     371031          0          0          0          0          0   PCI-MSI-edge      hpsa0
  69:         64          0          0     369089          0          0          0          0   PCI-MSI-edge      hpsa0
  70:        179          0          0          0      20790          0          0          0   PCI-MSI-edge      hpsa0
  71:         85          0          0          0          0      65244          0          0   PCI-MSI-edge      hpsa0
  72:        249          0          0          0          0          0      61309          0   PCI-MSI-edge      hpsa0
  73:        242          0          0          0          0          0          0      53981   PCI-MSI-edge      hpsa0
  74:          0          0          0          0          0          0          0          0   PCI-MSI-edge      eth0
  75:          6          0         20          0     903535       2705          0          0   PCI-MSI-edge      eth0-TxRx-0
  76:          6          0     903535         20          0       2705          0          0   PCI-MSI-edge      eth0-TxRx-1
  77:     903561          0          0          0          0       2705          0          0   PCI-MSI-edge      eth0-TxRx-2
  78:          6          0          0          0          0       2725     903535          0   PCI-MSI-edge      eth0-TxRx-3
  79:          6          0          0          0     903535       2705         20          0   PCI-MSI-edge      eth0-TxRx-4
  80:          6          0     903535          0          0       2705          0         20   PCI-MSI-edge      eth0-TxRx-5
  81:         86          0          0          0         20       2705          0     903455   PCI-MSI-edge      eth0-TxRx-6
  82:          6         20          0          0          0       2705     903535          0   PCI-MSI-edge      eth0-TxRx-7
  83:          0          0          0          0          0          0          0          0   PCI-MSI-edge      eth1
  84:          7          0     903535         20          0       2705          0          0   PCI-MSI-edge      eth1-TxRx-0
  85:        107          0          0          0          0       2705          0     903455   PCI-MSI-edge      eth1-TxRx-1
  86:          7          0          0          0          0       2725     903535          0   PCI-MSI-edge      eth1-TxRx-2
  87:          7          0          0          0     903535       2705         20          0   PCI-MSI-edge      eth1-TxRx-3
  88:          7          0     903535          0          0       2705          0         20   PCI-MSI-edge      eth1-TxRx-4
  89:         87          0          0          0         20       2705          0     903455   PCI-MSI-edge      eth1-TxRx-5
  90:          7         20          0          0          0       2705     903535          0   PCI-MSI-edge      eth1-TxRx-6
  91:          7          0         20          0     903535       2705          0          0   PCI-MSI-edge      eth1-TxRx-7
  92: 1563132111          0          0          0          0          0          0          0   PCI-MSI-edge      eth2-TxRx-0
  93:       3247  398335202          0          0          0          0          0          0   PCI-MSI-edge      eth2-TxRx-1
  94:       4681          0  411653663          0          0          0          0          0   PCI-MSI-edge      eth2-TxRx-2
  95:       6756          0          0  454354532          0          0          0          0   PCI-MSI-edge      eth2-TxRx-3
  96:       5505          0          0          0  391022444          0          0          0   PCI-MSI-edge      eth2-TxRx-4
  97:       4963          0          0          0          0  353505547          0          0   PCI-MSI-edge      eth2-TxRx-5
  98:       3434          0          0          0          0          0  396123077          0   PCI-MSI-edge      eth2-TxRx-6
  99:       5179          0          0          0          0          0          0  363277592   PCI-MSI-edge      eth2-TxRx-7
 100:          3          0     177365          1          0        337          0          0   PCI-MSI-edge      eth2
 101: 2154369752          0          0          0          0          0          0          0   PCI-MSI-edge      eth3-TxRx-0
 102:         11 4161200363          0          0          0          0          0          0   PCI-MSI-edge      eth3-TxRx-1
 103:         63          0 4263319316          0          0          0          0          0   PCI-MSI-edge      eth3-TxRx-2
 104:         11          0          0 4246099053          0          0          0          0   PCI-MSI-edge      eth3-TxRx-3
 105:         50          0          0          0 3702980025          0          0          0   PCI-MSI-edge      eth3-TxRx-4
 106:         43          0          0          0          0 3764516372          0          0   PCI-MSI-edge      eth3-TxRx-5
 107:         59          0          0          0          0          0 3834224668          0   PCI-MSI-edge      eth3-TxRx-6
 108:         27          0          0          0          0          0          0 3762967399   PCI-MSI-edge      eth3-TxRx-7
 109:          3          0          0          0          0          0          0          4   PCI-MSI-edge      eth3
 NMI:    1648058     299857     307650     294591     524422     396746     387824     440960   Non-maskable interrupts
 LOC: 3621212573 1878380193 1927318079 1959902316 1388530460 1495553213 1493434689 1471413791   Local timer interrupts
 SPU:          0          0          0          0          0          0          0          0   Spurious interrupts
 PMI:    1648058     299857     307650     294591     524422     396746     387824     440960   Performance monitoring interrupts
 IWI:          3          0          0          0          0          0          3          0   IRQ work interrupts
 RTR:          7          0          0          0          0          0          0          0   APIC ICR read retries
 RES:   54309538   75220665   33431078   40677661    3326965    8127609    8303768    8190846   Rescheduling interrupts
 CAL: 4294922491 4294893716 4294900660 4294895403 4294944733 4294933232 4294931023 4294927670   Function call interrupts
 TLB:    2776300    2302116    2319603    2444776     640386    1092192    1144436    1165548   TLB shootdowns
 TRM:          0          0          0          0          0          0          0          0   Thermal event interrupts
 THR:          0          0          0          0          0          0          0          0   Threshold APIC interrupts
 MCE:          0          0          0          0          0          0          0          0   Machine check exceptions
 MCP:       6043       6043       6043       6043       6043       6043       6043       6043   Machine check polls
 HYP:          0          0          0          0          0          0          0          0   Hypervisor callback interrupts
 ERR:          0
 MIS:          0


dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: Zombie process using 60% CPU

Post by dimka88 »

show perf top.
You use hyper threading?
hugleo
Posts: 59
Joined: 13 Apr 2016, 14:28

Re: Zombie process using 60% CPU

Post by hugleo »

Using ht

dmidecode | grep Count
Core Count: 4
Thread Count: 8

Code: Select all


Samples: 357K of event 'cycles', Event count (approx.): 40901450135
  13.11%  [kernel]              [k] pppoe_rcv                                                                                                                                   
   7.49%  [kernel]              [k] ipt_do_table                                                                                                                                
   3.72%  [kernel]              [k] _raw_spin_lock                                                                                                                              
   2.78%  [kernel]              [k] ixgbe_clean_rx_irq                                                                                                                          
   2.74%  [kernel]              [k] fib_table_lookup                                                                                                                            
   2.46%  [kernel]              [k] csum_partial_copy_generic                                                                                                                   
   2.15%  [kernel]              [k] memcpy                                                                                                                                      
   1.66%  [kernel]              [k] ixgbe_xmit_frame_ring                                                                                                                       
   1.55%  [kernel]              [k] __netif_receive_skb_core                                                                                                                    
   1.42%  [kernel]              [k] __dev_queue_xmit                                                                                                                            
   1.35%  [kernel]              [k] ip_route_input_noref                                                                                                                        
   1.31%  [kernel]              [k] timerqueue_add                                                                                                                              
   1.17%  [kernel]              [k] tcp_mt                                                                                                                                      
   1.17%  [kernel]              [k] ixgbe_poll                                                                                                                                  
   1.15%  [kernel]              [k] ip_finish_output                                                                                                                            
   1.08%  [kernel]              [k] dev_gro_receive                                                                                                                             
   1.06%  [kernel]              [k] nf_iterate                                                                                                                                  
   1.01%  [kernel]              [k] _raw_spin_lock_irqsave                                                                                                                      
   1.01%  [kernel]              [k] check_leaf.isra.6                                                                                                                           
   0.86%  [kernel]              [k] ip_rcv                                                                                                                                      
   0.82%  [kernel]              [k] cpu_startup_entry                                                                                                                           
   0.80%  [kernel]              [k] dev_hard_start_xmit                                                                                                                         
   0.80%  [kernel]              [k] kmem_cache_alloc                                                                                                                            
   0.76%  [kernel]              [k] menu_select                                                                                                                                 
   0.74%  [kernel]              [k] u32_classify                                                                                                                                
   0.73%  [kernel]              [k] put_page                                                                                                                                    
   0.69%  [kernel]              [k] ppp_xmit_process                                                                                                                            
   0.68%  [kernel]              [k] irq_entries_start                                                                                                                           
   0.67%  [kernel]              [k] tbf_dequeue                                                                                                                                 
   0.63%  [kernel]              [k] ip_forward                                                                                                                                  
   0.61%  [kernel]              [k] qdisc_dequeue_head                                                                                                                          
   0.60%  [kernel]              [k] __local_bh_enable_ip                                                                                                                        
   0.60%  [kernel]              [k] _raw_spin_lock_bh                                                                                                                           
   0.59%  [kernel]              [k] __inet_lookup_established                                                                                                                   
   0.59%  [kernel]              [k] apic_timer_interrupt                                                                                                                        
   0.58%  [kernel]              [k] inet_gro_receive                                                                                                                            
   0.55%  [kernel]              [k] __build_skb                                                                                                                                 
   0.51%  [kernel]              [k] rb_erase                                                                                                                                    
   0.50%  [kernel]              [k] kmem_cache_free                                                                                                                             
   0.48%  [kernel]              [k] _raw_spin_unlock_irqrestore                                                                                                                 
   0.45%  [kernel]              [k] int_sqrt                                                                                                                                    
   0.43%  [kernel]              [k] __netdev_pick_tx                                                                                                                            
   0.42%  [kernel]              [k] nf_hook_slow                                                                                                                                
                                                                                                           

dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: Zombie process using 60% CPU

Post by dimka88 »

I think ht visibility fake load and not recommended use on soft router. Maybe spread network card interrupts to CPU1,CPU2,CPU3,CPU4,CPU5,CPU6,CPU7. CPU0 interrupts use only for system needs.
vetheslav
Posts: 21
Joined: 14 Jul 2016, 12:45

Re: Zombie process using 60% CPU

Post by vetheslav »

Hello!
How is it possible to improve the productive capacity of pppoe_rcv?
dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: Zombie process using 60% CPU

Post by dimka88 »

Hi, what linux kernel version you are using? Maybe kernel update help you.
vetheslav
Posts: 21
Joined: 14 Jul 2016, 12:45

Re: Zombie process using 60% CPU

Post by vetheslav »

4.7.0-0.bpo.1-amd64
hugleo
Posts: 59
Joined: 13 Apr 2016, 14:28

Re: Zombie process using 60% CPU

Post by hugleo »

What about 4.8.last or 4.9? I'll try too. I see a lot of network fixes in last commits.
Maybe is acceptable since is balancing between the cores.

Another question. If I use ipoe will I got more performance? (considering arp cpu, vlans...)
Post Reply