Странное зависание ацеля

Questions related to general functionality
Post Reply
bodigard
Posts: 24
Joined: 24 Mar 2015, 04:37

Странное зависание ацеля

Post by bodigard »

Доброго времени суток !

ацель используеться в качестве ipoe

возникла проблема, странно повисает ацель, accel-cmd show stat, accel-cmd show sessions и т.д. просто повисают ничего не показывая
при этом новые сессии не поднимаются, а те которые уже были вроде как нормально работают, при этом ацель перестаёт писать логи

в системном логе при этом наблюдается такое

Code: Select all

Dec 13 18:45:09 ipoe1 kernel: [1910283.580105] INFO: task accel-pppd:26476 blocked for more than 120 seconds.
Dec 13 18:45:09 ipoe1 kernel: [1910283.580139] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 13 18:45:09 ipoe1 kernel: [1910283.580166] accel-pppd      D f3d45710     0 26476      1 0x00000000
Dec 13 18:45:09 ipoe1 kernel: [1910283.580181]  f762aed0 00200082 d6e74710 f3d45710 00000000 00000000 d72658c8 c1482ac0
Dec 13 18:45:09 ipoe1 kernel: [1910283.580206]  f762b080 c1482ac0 f127b180 d6e74710 dc21e518 f3d45710 d6e74710 00000000
Dec 13 18:45:09 ipoe1 kernel: [1910283.580226]  f84c5be0 f84d361c 00001029 dc21e518 f7842200 00000000 dc21e490 f84a8428
Dec 13 18:45:09 ipoe1 kernel: [1910283.580247] Call Trace:
Dec 13 18:45:09 ipoe1 kernel: [1910283.580316]  [<f84c5be0>] ? __ext4_handle_dirty_metadata+0xda/0x119 [ext4]
Dec 13 18:45:09 ipoe1 kernel: [1910283.580355]  [<f84a8428>] ? ext4_mark_iloc_dirty+0x40a/0x4d3 [ext4]
Dec 13 18:45:09 ipoe1 kernel: [1910283.580388]  [<f84a848a>] ? ext4_mark_iloc_dirty+0x46c/0x4d3 [ext4]
Dec 13 18:45:09 ipoe1 kernel: [1910283.580406]  [<c12c4cfb>] ? __mutex_lock_common.isra.5+0xdd/0x12d
Dec 13 18:45:09 ipoe1 kernel: [1910283.580418]  [<c12c4c12>] ? mutex_lock+0x15/0x21
Dec 13 18:45:09 ipoe1 kernel: [1910283.580429]  [<c122be22>] ? rtnetlink_rcv+0x9/0x1c
Dec 13 18:45:09 ipoe1 kernel: [1910283.580441]  [<c123c195>] ? netlink_unicast+0xc0/0x115
Dec 13 18:45:09 ipoe1 kernel: [1910283.580452]  [<c123c432>] ? netlink_sendmsg+0x248/0x274
Dec 13 18:45:09 ipoe1 kernel: [1910283.580465]  [<c1213e05>] ? sock_sendmsg+0xa8/0xc2
Dec 13 18:45:09 ipoe1 kernel: [1910283.580479]  [<c10970be>] ? generic_file_buffered_write+0x18d/0x1dd
Dec 13 18:45:09 ipoe1 kernel: [1910283.580497]  [<c101ecab>] ? __default_send_IPI_dest_field+0x2f/0x4c
Dec 13 18:45:09 ipoe1 kernel: [1910283.580511]  [<c11671bc>] ? _copy_from_user+0x28/0x47
Dec 13 18:45:09 ipoe1 kernel: [1910283.580521]  [<c12c45ba>] ? _cond_resched+0x5/0x18
Dec 13 18:45:09 ipoe1 kernel: [1910283.580532]  [<c11671bc>] ? _copy_from_user+0x28/0x47
Dec 13 18:45:09 ipoe1 kernel: [1910283.580543]  [<c121c0c5>] ? verify_iovec+0x48/0x7f
Dec 13 18:45:09 ipoe1 kernel: [1910283.580554]  [<c1214cf0>] ? ___sys_sendmsg.part.13+0x14f/0x1e1
Dec 13 18:45:09 ipoe1 kernel: [1910283.580567]  [<c10ad2b5>] ? do_wp_page+0x2f3/0x5fd
Dec 13 18:45:09 ipoe1 kernel: [1910283.580579]  [<c10aee30>] ? handle_pte_fault+0x850/0x8c9
Dec 13 18:45:09 ipoe1 kernel: [1910283.580591]  [<c12c559e>] ? _raw_spin_unlock_irqrestore+0xb/0xc
Dec 13 18:45:09 ipoe1 kernel: [1910283.580603]  [<c103240e>] ? try_to_wake_up+0x14b/0x155
Dec 13 18:45:09 ipoe1 kernel: [1910283.580613]  [<c1029234>] ? kmap_atomic_prot+0xcc/0xe0
Dec 13 18:45:09 ipoe1 kernel: [1910283.580624]  [<c10af110>] ? handle_mm_fault+0x1eb/0x201
Dec 13 18:45:09 ipoe1 kernel: [1910283.580635]  [<c102a399>] ? should_resched+0x5/0x1e
Dec 13 18:45:09 ipoe1 kernel: [1910283.580645]  [<c12c45ba>] ? _cond_resched+0x5/0x18
Dec 13 18:45:09 ipoe1 kernel: [1910283.580655]  [<c11671bc>] ? _copy_from_user+0x28/0x47
Dec 13 18:45:09 ipoe1 kernel: [1910283.580666]  [<c12158f1>] ? __sys_sendmsg+0x2b/0x48
Dec 13 18:45:09 ipoe1 kernel: [1910283.580677]  [<c1215de7>] ? sys_socketcall+0x181/0x1cd
Dec 13 18:45:09 ipoe1 kernel: [1910283.580688]  [<c12c8195>] ? vmalloc_fault+0x87/0x87
Dec 13 18:45:09 ipoe1 kernel: [1910283.580699]  [<c12c576c>] ? syscall_call+0x7/0x7
Dec 13 18:45:09 ipoe1 kernel: [1910283.580709] INFO: task accel-pppd:29989 blocked for more than 120 seconds.
Dec 13 18:45:09 ipoe1 kernel: [1910283.580733] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Dec 13 18:45:09 ipoe1 kernel: [1910283.580759] accel-pppd      D f3d45710     0 29989      1 0x00000000
Dec 13 18:45:09 ipoe1 kernel: [1910283.580771]  dd690dd0 00200082 d72af638 f3d45710 00000000 00000000 d72658c8 c1482ac0
Dec 13 18:45:09 ipoe1 kernel: [1910283.580791]  dd690f80 c1482ac0 f127b180 d72af638 dc21e518 f3d45710 d72af638 00000000
Dec 13 18:45:09 ipoe1 kernel: [1910283.580811]  f84c5be0 f84d361c 00001029 dc21e518 f7842200 00000000 dc21e490 f84a8428
Dec 13 18:45:09 ipoe1 kernel: [1910283.580831] Call Trace:
Dec 13 18:45:09 ipoe1 kernel: [1910283.580871]  [<f84c5be0>] ? __ext4_handle_dirty_metadata+0xda/0x119 [ext4]
Dec 13 18:45:09 ipoe1 kernel: [1910283.580905]  [<f84a8428>] ? ext4_mark_iloc_dirty+0x40a/0x4d3 [ext4]
Dec 13 18:45:09 ipoe1 kernel: [1910283.580937]  [<f84a848a>] ? ext4_mark_iloc_dirty+0x46c/0x4d3 [ext4]
Dec 13 18:45:09 ipoe1 kernel: [1910283.580949]  [<c12c4cfb>] ? __mutex_lock_common.isra.5+0xdd/0x12d
Dec 13 18:45:09 ipoe1 kernel: [1910283.580961]  [<c12c4c12>] ? mutex_lock+0x15/0x21
Dec 13 18:45:09 ipoe1 kernel: [1910283.580970]  [<c122be22>] ? rtnetlink_rcv+0x9/0x1c
Dec 13 18:45:09 ipoe1 kernel: [1910283.580980]  [<c123c195>] ? netlink_unicast+0xc0/0x115
Dec 13 18:45:09 ipoe1 kernel: [1910283.580990]  [<c123c432>] ? netlink_sendmsg+0x248/0x274
Dec 13 18:45:09 ipoe1 kernel: [1910283.581001]  [<c1213e05>] ? sock_sendmsg+0xa8/0xc2
Dec 13 18:45:09 ipoe1 kernel: [1910283.581012]  [<c10970be>] ? generic_file_buffered_write+0x18d/0x1dd
Dec 13 18:45:09 ipoe1 kernel: [1910283.581027]  [<c101ecab>] ? __default_send_IPI_dest_field+0x2f/0x4c
Dec 13 18:45:09 ipoe1 kernel: [1910283.581039]  [<c11671bc>] ? _copy_from_user+0x28/0x47
Dec 13 18:45:09 ipoe1 kernel: [1910283.581049]  [<c12c45ba>] ? _cond_resched+0x5/0x18
Dec 13 18:45:09 ipoe1 kernel: [1910283.581059]  [<c11671bc>] ? _copy_from_user+0x28/0x47
Dec 13 18:45:09 ipoe1 kernel: [1910283.581069]  [<c121c0c5>] ? verify_iovec+0x48/0x7f
Dec 13 18:45:09 ipoe1 kernel: [1910283.581079]  [<c1214cf0>] ? ___sys_sendmsg.part.13+0x14f/0x1e1
Dec 13 18:45:09 ipoe1 kernel: [1910283.581091]  [<c10ad2b5>] ? do_wp_page+0x2f3/0x5fd
Dec 13 18:45:09 ipoe1 kernel: [1910283.581102]  [<c10aee30>] ? handle_pte_fault+0x850/0x8c9
Dec 13 18:45:09 ipoe1 kernel: [1910283.581114]  [<c12c559e>] ? _raw_spin_unlock_irqrestore+0xb/0xc
Dec 13 18:45:09 ipoe1 kernel: [1910283.581124]  [<c103240e>] ? try_to_wake_up+0x14b/0x155
Dec 13 18:45:09 ipoe1 kernel: [1910283.581133]  [<c1029234>] ? kmap_atomic_prot+0xcc/0xe0
Dec 13 18:45:09 ipoe1 kernel: [1910283.581144]  [<c10af110>] ? handle_mm_fault+0x1eb/0x201
Dec 13 18:45:09 ipoe1 kernel: [1910283.581154]  [<c102a399>] ? should_resched+0x5/0x1e
Dec 13 18:45:09 ipoe1 kernel: [1910283.581164]  [<c12c45ba>] ? _cond_resched+0x5/0x18
Dec 13 18:45:09 ipoe1 kernel: [1910283.581174]  [<c11671bc>] ? _copy_from_user+0x28/0x47
Dec 13 18:45:09 ipoe1 kernel: [1910283.581185]  [<c12158f1>] ? __sys_sendmsg+0x2b/0x48
Dec 13 18:45:09 ipoe1 kernel: [1910283.581196]  [<c1215de7>] ? sys_socketcall+0x181/0x1cd
Dec 13 18:45:09 ipoe1 kernel: [1910283.581206]  [<c12c8195>] ? vmalloc_fault+0x87/0x87
Dec 13 18:45:09 ipoe1 kernel: [1910283.581216]  [<c12c576c>] ? syscall_call+0x7/0x7
также в этот момент судя по top-у нагрузки нет никакой, памяти свободной тоже полно, но тот-же top говорит что у меня в системе ~350 зомби процессов
кстати места на дисках тоже полно и ротация логов настроена

читал про ошибку INFO: task ХХХ blocked for more than 120 seconds. и по рекомендациям сделал
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
но к сожалению не помогло

система

Code: Select all

uname -a
Linux ipoe1 3.2.0-4-686-pae #1 SMP Debian 3.2.65-1 i686 GNU/Linux
ацель

Code: Select all

[2015-12-14 09:05:21]:   msg: accel-ppp version 1.9.0
подскажите, куда копать ?
Post Reply