accelppp exits unexpectedly with log error md:epoll_wait

Questions related to general functionality
exopedro
Posts: 8
Joined: 23 Jun 2020, 15:17

accelppp exits unexpectedly with log error md:epoll_wait

Post by exopedro »

Hi,

We are hitting an error with accel-ppp latest stable version (1.12.0) with nonprofit ISP exo.cat

The same accelppp server uses L2TP for subscribers with wifi network and PPPoE for subscribers with FTTH network (there we use vlan_mon custom kernel module from accelppp)

The problem is that accelppp suddenly exits and systemd reloads it, so we suffer a very small cut that these days people note it because they are heavinly using their internet for the videoconference tools.

It's difficult to see the error because after accel exits, the debug.log is overriden we still don't know how make it more "append". Looking at the logs and so on we saw that exits in the code sometimes are not reported. So we tracked all exits this way [1], and we got the stderr from accelppp this way in systemd [2]

Our error is:

`md.c: Debug-eXO: (). Function md_thread(). n < 0. errno != EINTR. Exiting` in our code (just adding stderr before exits) [3]

which corresponds to this line in the official code [4]

Accelppp is fine most of the time according to `accel-cmd show stat` [5]

this is our config file /etc/accel-ppp.conf [6]

we build accelppp with this install script [7] on top of a debian 10 stable that is in a virtual machine (KVM) in a proxmox.org cluster with resources shared with other unrelated services. This one has: 4 vCPUs and 2 GB of RAM

the backbone interface has this ethtool options [8]

cat /proc/interrupts [9]

the load average is, in general: 0.00, 0.00, 0.00

with `htop` program, in general, all 4 vCPUs are around 0-4 % and some of them sometimes do 10%. accelppp uses around 0 and 1%. The most important process is node exporter with that 10% sometimes. The prometheus scrape_interval is 5s. The process is monitored with prometheus node exporter and with a grafana dashboard, so in case you want a specific metric we can get it, or show screen capture.

We think that the service requires more CPU time so we would like to:

- increase cpu units [10]
- we would like to do `ethtool -G ens18 rx 4096 tx 4096` as suggested by [11], but we cannot make it, it says "cannot set device ring parameters: operation not supported"

It would be nice to have a torture testbed to reproduce this in the lab and apply parameters in a more freely way. Do we have something? Any help is appreciated.

Thanks,
Pedro

[1] https://gitlab.com/guifi-exo/accel-ppp/ ... 38af112409
[2] (1) /usr/sbin/accel-pppd without `-d` (2) `Type=simple` https://gitlab.com/guifi-exo/accel-ppp/ ... pp.service
[3] https://gitlab.com/guifi-exo/accel-ppp/ ... n/md.c#L80
[4] https://github.com/xebd/accel-ppp/blob/ ... n/md.c#L77
[5] http://paste.debian.net/1153497/
[6] http://paste.debian.net/1153496/
[7] https://gitlab.com/guifi-exo/accel-ppp/ ... git-exo.sh
[8] (with proxmox queues=4 in a virtio iface)
auto ens18
iface ens18 inet manual
pre-up /sbin/ethtool -K ens18 tx off rx off && /sbin/ethtool -L ens18 combined 4
[9] http://paste.debian.net/1153498/
[10] https://serverfault.com/questions/20547 ... mox/364809
[11] viewtopic.php?f=9&t=2689#p7110
exopedro
Posts: 8
Joined: 23 Jun 2020, 15:17

Re: accelppp exits unexpectedly with log error md:epoll_wait

Post by exopedro »

we followed all steps to monitor the error in the link you provided

in order to get the backtrace (which is not documented in accel-ppp docs) we used the following script (latest version)

Code: Select all

#!/bin/sh

# put this in tmux
# thanks pespin

outfile="/root/accelppp_gdb.log"

# generic debug in accelppp -> src https://accel-ppp.readthedocs.io/en/latest/debugging/index.html
# about attach -> src http://sourceware.org/gdb/onlinedocs/gdb/Attach.html

if [ $(id -u) -ne 0 ]
  then echo "Please run as root"
  exit
fi

log_header="================== $(date +'%Y-%m-%d-%H-%M-%S-%N') =================="
echo "$log_header" >> "$outfile"

# appropriate order to send stderr to stdout, and stdout to append file
#   -> src https://stackoverflow.com/questions/876239/how-to-redirect-and-append-both-stdout-and-stderr-to-a-file-with-bash/876242#876242
gdb -ex 'set breakpoint pending on' -ex 'set confirm off' \
  -ex 'set pagination off' -ex 'b _exit' -ex 'c' -ex 'bt full' \
  -ex 'q' -p "$(pidof accel-pppd)" >> "$outfile" 2>&1
the Debug-eXO function is this line we added (high detail):

Code: Select all

fprintf(stderr, "md.c: Debug-eXO: (). Function md_run(). pthread_create(&md_thr, NULL, md_thread, NULL) => 0. Exiting. errno = %s\n", strerror(errno));
and we hit this line that is shown in systemd journal (our systemd service version):

Code: Select all

Jul 05 18:10:12 bng accelppp-exo[20044]: md.c: Debug-eXO: (). Function md_thread(). n < 0. errno = Bad file descriptor
Jul 05 18:10:14 bng systemd[1]: accel-ppp.service: Main process exited, code=exited, status=255/EXCEPTION
Jul 05 18:10:14 bng systemd[1]: accel-ppp.service: Failed with result 'exit-code'.
Jul 05 18:10:14 bng systemd[1]: accel-ppp.service: Service RestartSec=100ms expired, scheduling restart.
Jul 05 18:10:14 bng systemd[1]: accel-ppp.service: Scheduled restart job, restart counter is at 5.
Jul 05 18:10:14 bng systemd[1]: Stopped Accel-PPP.
Jul 05 18:10:14 bng systemd[1]: Started Accel-PPP.
accel-pppd backtrace:

Code: Select all

================== 2020-07-03-10-25-00-413179131 ==================
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 20044
[New LWP 20045]
[New LWP 20046]
[New LWP 20047]
[New LWP 20048]
[New LWP 20049]
[New LWP 20050]
[New LWP 20051]
[New LWP 20052]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fb197f154cc in __GI___sigtimedwait (set=set@entry=0x7ffc8bee2740, info=info@entry=0x7ffc8bee2650, timeout=timeout@entry=0x0)
    at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
29      ../sysdeps/unix/sysv/linux/sigtimedwait.c: No such file or directory.
Breakpoint 1 at 0x7fb197fa39a0: file ../sysdeps/unix/sysv/linux/_exit.c, line 27.
Continuing.

Thread 5 "accel-pppd" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fb1968f0700 (LWP 20048)]
0x00007fb197e5efa3 in find_pd (ses=0x7fb17c00bcc8) at /usr/local/src/accel-ppp/accel-pppd/radius/radius.c:731
731             list_for_each_entry(pd, &ses->pd_list, entry) {
#0  0x00007fb197e5efa3 in find_pd (ses=0x7fb17c00bcc8) at /usr/local/src/accel-ppp/accel-pppd/radius/radius.c:731
        pd = 0x0
        rpd = 0x7fb18c00f1c8
#1  0x00007fb197e5e9b2 in ses_finishing (ses=0x7fb17c00bcc8) at /usr/local/src/accel-ppp/accel-pppd/radius/radius.c:608
        rpd = 0x55d16afcf670
        fr6 = 0x7fb197a88526 <ev_ses_finishing+39>
        fr = 0x7fb18c0386b8
#2  0x00007fb1984cfb44 in triton_event_fire (ev_id=3, arg=0x7fb17c00bcc8) at /usr/local/src/accel-ppp/accel-pppd/triton/event.c:103
        ev = 0x55d16afcf670
        h = 0x55d16afddde0
#3  0x000055d169e5f300 in ap_session_terminate (ses=0x7fb17c00bcc8, cause=1, hard=0) at /usr/local/src/accel-ppp/accel-pppd/session.c:302
No locals.
#4  0x000055d169e68b92 in lcp_recv (h=0x7fb18c0386e0) at /usr/local/src/accel-ppp/accel-pppd/ppp/ppp_lcp.c:817
        hdr = 0x7fb178051d68
        lcp = 0x7fb18c0386b8
        r = 32689
        term_msg = 0x55d16afcf028 "\310\360\374j\321U"
#5  0x000055d169e63811 in ppp_chan_read (h=0x7fb17c00bdf8) at /usr/local/src/accel-ppp/accel-pppd/ppp/ppp.c:423
        ppp = 0x7fb17c00bcc8
        ppp_h = 0x7fb18c0386e0
        proto = 49185
#6  0x00007fb1984cbcaa in ctx_thread (ctx=0x7fb17804e138) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:251
        h = 0x7fb18c021d38
        t = 0x7fb17c00bcc8
        call = 0x7fb197e92b00 <l2tp_ctx_switch>
        tt = 140400266361928
        events = 1
#7  0x00007fb1984cba56 in triton_thread (thread=0x55d16b0125f8) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:192
        set = {__val = {516, 0 <repeats 15 times>}}
        sig = 10
        need_free = 0
        stack = 0x0
#8  0x00007fb19849bfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140400711894784, -8357117975546649785, 140722656126414, 140722656126415, 140400711894784, 0, 8315510127604179783, 8315542574019420999}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#9  0x00007fb197fd64cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.
Detaching from program: /usr/sbin/accel-pppd, process 20044
[Inferior 1 (process 20044) detached]
dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: accelppp exits unexpectedly with log error md:epoll_wait

Post by dimka88 »

Hi, did you have a chance to build and run accel-ppp from the master branch https://github.com/accel-ppp/accel-ppp
Did you save debug logs?
exopedro
Posts: 8
Joined: 23 Jun 2020, 15:17

Re: accelppp exits unexpectedly with log error md:epoll_wait

Post by exopedro »

Hi again,

Yes, we have debug logs. We can send to you privately.

We will try to run master, but we would like to try it in testing before entering production. Would you like to specify a certain commit to do git checkout there?

These days we are running accel-ppp in production with debug symbols and we started to have SIGABRTs, we have one of them with high detail of debug (see Error1)

We also discovered that the accelppp's VM has 4 vCPUs and accelppp config thread-count is 4, but we are seeing 8 threads (in gdb and htop programs and accel-cmd show stat), we tried to reduce the number of threads without success.

Error1 (with SIGABRT)

this is syslog

Code: Select all

Jul 12 03:28:11 bng kernel: [6025160.365840] maskediface: renamed from ppp24
Jul 12 03:28:22 bng systemd-udevd[8587]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:28:22 bng kernel: [6025170.861240] maskediface: renamed from ppp24
Jul 12 03:28:33 bng systemd-udevd[8597]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:28:33 bng kernel: [6025182.100235] maskediface: renamed from ppp24
Jul 12 03:28:44 bng systemd-udevd[8606]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:28:44 bng kernel: [6025192.598966] maskediface: renamed from ppp24
Jul 12 03:28:55 bng systemd-udevd[8615]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:28:55 bng kernel: [6025203.840547] maskediface: renamed from ppp24
Jul 12 03:29:05 bng systemd-udevd[8624]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:29:05 bng kernel: [6025214.341482] maskediface: renamed from ppp24
Jul 12 03:29:16 bng systemd-udevd[8633]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:29:26 bng kernel: [6025235.072464] maskediface: renamed from ppp24
Jul 12 03:29:37 bng systemd-udevd[8644]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:29:37 bng kernel: [6025245.593549] maskediface: renamed from ppp24
Jul 12 03:29:48 bng systemd-udevd[8653]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:29:48 bng kernel: [6025256.836815] maskediface: renamed from ppp24
Jul 12 03:29:59 bng systemd-udevd[8662]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:29:59 bng kernel: [6025268.073242] maskediface: renamed from ppp24
Jul 12 03:30:09 bng systemd-udevd[8671]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:30:10 bng kernel: [6025278.567543] maskediface: renamed from ppp24
Jul 12 03:30:20 bng systemd-udevd[8680]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:30:20 bng kernel: [6025289.056366] maskediface: renamed from ppp24
Jul 12 03:30:30 bng systemd-udevd[8689]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:30:31 bng kernel: [6025299.557446] maskediface: renamed from ppp24
Jul 12 03:30:41 bng systemd-udevd[8698]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:30:41 bng kernel: [6025310.043541] maskediface: renamed from ppp24
Jul 12 03:30:51 bng systemd-udevd[8707]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:30:52 bng kernel: [6025320.547623] maskediface: renamed from ppp24
Jul 12 03:31:02 bng systemd-udevd[8716]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:31:12 bng kernel: [6025341.276127] maskediface: renamed from ppp24
Jul 12 03:31:23 bng systemd-udevd[8727]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:31:23 bng kernel: [6025351.785156] maskediface: renamed from ppp24
Jul 12 03:31:34 bng systemd-udevd[8736]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:31:34 bng kernel: [6025363.007930] maskediface: renamed from ppp24
Jul 12 03:31:45 bng systemd-udevd[8746]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:31:45 bng kernel: [6025374.272965] maskediface: renamed from ppp24
Jul 12 03:31:56 bng systemd-udevd[8755]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:31:56 bng kernel: [6025385.487809] maskediface: renamed from ppp24
Jul 12 03:32:08 bng systemd-udevd[8765]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:32:08 bng kernel: [6025396.723069] maskediface: renamed from ppp24
Jul 12 03:32:08 bng kernel: [6025396.743921] ppp24: renamed from maskediface
Jul 12 03:32:18 bng kernel: [6025406.972861] maskediface: renamed from ppp24
Jul 12 03:32:29 bng systemd-udevd[8776]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:32:29 bng kernel: [6025418.191512] maskediface: renamed from ppp24
Jul 12 03:32:40 bng systemd-udevd[8785]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:32:40 bng kernel: [6025429.418808] maskediface: renamed from ppp24
Jul 12 03:32:52 bng systemd-udevd[8795]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:32:52 bng kernel: [6025440.648030] maskediface: renamed from ppp24
Jul 12 03:33:02 bng systemd-udevd[8804]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 12 03:33:02 bng kernel: [6025451.147744] maskediface: renamed from ppp24
Jul 12 03:33:04 bng systemd[1]: accel-ppp.service: Main process exited, code=killed, status=6/ABRT
Jul 12 03:33:04 bng systemd[1]: accel-ppp.service: Failed with result 'signal'.
Jul 12 03:33:04 bng systemd[1]: accel-ppp.service: Service RestartSec=100ms expired, scheduling restart.
Jul 12 03:33:04 bng systemd[1]: accel-ppp.service: Scheduled restart job, restart counter is at 6.
Jul 12 03:33:04 bng systemd[1]: Stopped Accel-PPP.
Jul 12 03:33:04 bng systemd[1]: Started Accel-PPP.
Now we are running gdb with thread apply all bt full

Code: Select all

================== 2020-07-08-00-24-54-351292934 ===================
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 27776
[New LWP 27777]
[New LWP 27778]
[New LWP 27779]
[New LWP 27780]
[New LWP 27781]
[New LWP 27782]
[New LWP 27783]
[New LWP 27784]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f4f2e5964cc in __GI___sigtimedwait (set=set@entry=0x7ffef54ef970, info=info@entry=0x7ffef54ef880, timeout=timeout@entry=0x0)
    at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
29	../sysdeps/unix/sysv/linux/sigtimedwait.c: No such file or directory.
Breakpoint 1 at 0x7f4f2e6249a0: file ../sysdeps/unix/sysv/linux/_exit.c, line 27.
Continuing.

Thread 2 "accel-pppd" received signal SIGABRT, Aborted.
[Switching to Thread 0x7f4f2e0f5700 (LWP 27777)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

Thread 9 (Thread 0x7f4f2c46d700 (LWP 27784)):
#0  0x00007f4f2e6577ef in epoll_wait (epfd=9, events=0x55883b8d0748, maxevents=64, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
        resultvar = 18446744073709551612
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  0x00007f4f2eb4b93e in timer_thread (arg=0x0) at /usr/local/src/accel-ppp/accel-pppd/triton/timer.c:82
        i = 1
        n = 1
        r = 1
        t = 0x7f4f0c0257e8
        set = {__val = {18446744067266838271, 18446744073709551615 <repeats 15 times>}}
#2  0x00007f4f2eb1cfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139978021984000, -6510256976078385386, 140733014014014, 140733014014015, 139978021984000, 0, 6573200779909713686, 6573204482220674838}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#3  0x00007f4f2e6574cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 8 (Thread 0x7f4f2cc6e700 (LWP 27783)):
#0  0x00007f4f2e6577ef in epoll_wait (epfd=8, events=0x55883b8d00c8, maxevents=64, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
        resultvar = 18446744073709551612
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  0x00007f4f2eb4ab6d in md_thread (arg=0x0) at /usr/local/src/accel-ppp/accel-pppd/triton/md.c:75
        i = 1
        n = 1
        r = 1
        h = 0x7f4f20031118
        set = {__val = {18446744067266838271, 18446744073709551615 <repeats 15 times>}}
#2  0x00007f4f2eb1cfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139978030376704, -6510256976078385386, 140733014014014, 140733014014015, 139978030376704, 0, 6573199679861214998, 6573204482220674838}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#3  0x00007f4f2e6574cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 7 (Thread 0x7f4f2cd6f700 (LWP 27782)):
#0  0x00007f4f2e5964cc in __GI___sigtimedwait (set=set@entry=0x7f4f2cd6ed10, info=info@entry=0x7f4f2cd6ec40, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
        __arg4 = 8
        __arg2 = 139978031426624
        _a3 = 0
        _a1 = 139978031426832
        resultvar = <optimized out>
        __arg3 = 0
        __arg1 = 139978031426832
        _a4 = 8
        _a2 = 139978031426624
        resultvar = <optimized out>
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
        result = <optimized out>
        sc_ret = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        sc_cancel_oldtype = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  0x00007f4f2eb272bc in __sigwait (set=0x7f4f2cd6ed10, sig=0x7f4f2cd6ed00) at ../sysdeps/unix/sysv/linux/sigwait.c:28
        si = {si_signo = 1, si_errno = -1, si_code = 0, __pad0 = 0, _sifields = {_pad = {201561792, 32591, 201561720, 32591, 825110577, 775172398, 3618609, 32591, 0, 0, 201536056, 32591, 0, 0, 999293944, 21896, 783631152, 32591, 999610984, 21896, 752282848, 32591, 717405696, 2122344702, 752282848, 32591, 783597950, 32591}, _kill = {si_pid = 201561792, si_uid = 32591}, _timer = {si_tid = 201561792, si_overrun = 32591, si_sigval = {sival_int = 201561720, sival_ptr = 0x7f4f0c039678}}, _rt = {si_pid = 201561792, si_uid = 32591, si_sigval = {sival_int = 201561720, sival_ptr = 0x7f4f0c039678}}, _sigchld = {si_pid = 201561792, si_uid = 32591, si_status = 201561720, si_utime = 3329340098997006385, si_stime = 139977282762545}, _sigfault = {si_addr = 0x7f4f0c0396c0, si_addr_lsb = -27016, _bounds = {_addr_bnd = {_lower = 0x2e34312e312e3031, _upper = 0x7f4f00373731}, _pkey = 825110577}}, _sigpoll = {si_band = 139977480705728, si_fd = 201561720}, _sigsys = {_call_addr = 0x7f4f0c0396c0, _syscall = 201561720, _arch = 32591}}}
        ret = <optimized out>
#2  0x00007f4f2eb4c946 in triton_thread (thread=0x55883b94da58) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:171
        set = {__val = {516, 0 <repeats 15 times>}}
        sig = 10
        need_free = 0
        stack = 0x0
#3  0x00007f4f2eb1cfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139978031429376, -6510256976078385386, 140733014013950, 140733014013951, 139978031429376, 0, 6573199816763297558, 6573204482220674838}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#4  0x00007f4f2e6574cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 6 (Thread 0x7f4f2ce70700 (LWP 27781)):
#0  0x00007f4f2e5964cc in __GI___sigtimedwait (set=set@entry=0x7f4f2ce6fd10, info=info@entry=0x7f4f2ce6fc40, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
        __arg4 = 8
        __arg2 = 139978032479296
        _a3 = 0
        _a1 = 139978032479504
        resultvar = <optimized out>
        __arg3 = 0
        __arg1 = 139978032479504
        _a4 = 8
        _a2 = 139978032479296
        resultvar = <optimized out>
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
        result = <optimized out>
        sc_ret = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        sc_cancel_oldtype = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  0x00007f4f2eb272bc in __sigwait (set=0x7f4f2ce6fd10, sig=0x7f4f2ce6fd00) at ../sysdeps/unix/sysv/linux/sigwait.c:28
        si = {si_signo = 1, si_errno = -1, si_code = 0, __pad0 = 0, _sifields = {_pad = {201561792, 32591, 201561720, 32591, 825110577, 775172398, 3618609, 32591, 0, 0, 537198584, 32591, 0, 0, 999610984, 21896, 783631152, 32591, 999610424, 21896, 753335520, 32591, 717405696, 2122344702, 753335520, 32591, 783597950, 32591}, _kill = {si_pid = 201561792, si_uid = 32591}, _timer = {si_tid = 201561792, si_overrun = 32591, si_sigval = {sival_int = 201561720, sival_ptr = 0x7f4f0c039678}}, _rt = {si_pid = 201561792, si_uid = 32591, si_sigval = {sival_int = 201561720, sival_ptr = 0x7f4f0c039678}}, _sigchld = {si_pid = 201561792, si_uid = 32591, si_status = 201561720, si_utime = 3329340098997006385, si_stime = 139977282762545}, _sigfault = {si_addr = 0x7f4f0c0396c0, si_addr_lsb = -27016, _bounds = {_addr_bnd = {_lower = 0x2e34312e312e3031, _upper = 0x7f4f00373731}, _pkey = 825110577}}, _sigpoll = {si_band = 139977480705728, si_fd = 201561720}, _sigsys = {_call_addr = 0x7f4f0c0396c0, _syscall = 201561720, _arch = 32591}}}
        ret = <optimized out>
#2  0x00007f4f2eb4c946 in triton_thread (thread=0x55883b94d828) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:171
        set = {__val = {516, 0 <repeats 15 times>}}
        sig = 10
        need_free = 0
        stack = 0x0
#3  0x00007f4f2eb1cfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139978032482048, -6510256976078385386, 140733014013950, 140733014013951, 139978032482048, 0, 6573199403909566230, 6573204482220674838}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#4  0x00007f4f2e6574cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 5 (Thread 0x7f4f2cf71700 (LWP 27780)):
#0  0x00007f4f2eb265d7 in __close (fd=313) at ../sysdeps/unix/sysv/linux/close.c:27
        resultvar = 0
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  0x00007f4f2eb4af65 in triton_md_unregister_handler (ud=0x7f4f24032c68, c=1) at /usr/local/src/accel-ppp/accel-pppd/triton/md.c:146
        h = 0x7f4f0c013ff8
#2  0x0000558839c972fe in destablish_ppp (ppp=0x7f4f24032b18) at /usr/local/src/accel-ppp/accel-pppd/ppp/ppp.c:290
        ifr = {ifr_ifrn = {ifrn_name = "i0068c0\000\000\000\000\000\000\000\000"}, ifr_ifru = {ifru_addr = {sa_family = 28784, sa_data = "p24\000\000\000\030\221\000 \002\000\000"}, ifru_dstaddr = {sa_family = 28784, sa_data = "p24\000\000\000\030\221\000 \002\000\000"}, ifru_broadaddr = {sa_family = 28784, sa_data = "p24\000\000\000\030\221\000 \002\000\000"}, ifru_netmask = {sa_family = 28784, sa_data = "p24\000\000\000\030\221\000 \002\000\000"}, ifru_hwaddr = {sa_family = 28784, sa_data = "p24\000\000\000\030\221\000 \002\000\000"}, ifru_flags = 28784, ifru_ivalue = 846229616, ifru_mtu = 846229616, ifru_map = {mem_start = 224184529008, mem_end = 9126842648, base_addr = 1, irq = 0 '\000', dma = 0 '\000', port = 0 '\000'}, ifru_slave = "ppp24\000\000\000\030\221\000 \002\000\000", ifru_newname = "ppp24\000\000\000\030\221\000 \002\000\000", ifru_data = 0x3432707070, ifru_settings = {type = 846229616, size = 52, ifs_ifsu = {raw_hdlc = 0x220009118, cisco = 0x220009118, fr = 0x220009118, fr_pvc = 0x220009118, fr_pvc_info = 0x220009118, sync = 0x220009118, te1 = 0x220009118}}}}
        uc = 0x0
#3  0x0000558839c98040 in ppp_terminate (ses=0x7f4f24032b18, hard=1) at /usr/local/src/accel-ppp/accel-pppd/ppp/ppp.c:587
        ppp = 0x7f4f24032b18
        n = 0x558839c92822 <_time+40>
        d = 0x5bf0f7
        s = 0
#4  0x0000558839c932ce in ap_session_terminate (ses=0x7f4f24032b18, cause=6, hard=1) at /usr/local/src/accel-ppp/accel-pppd/session.c:298
No locals.
#5  0x00007f4f2e5173c1 in apses_stop (data=0x6) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:1328
        sess = 0x7f4f24032a18
        cause = 6
#6  0x00007f4f2eb4cd6b in ctx_thread (ctx=0x7f4f0c0331c8) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:272
        h = 0x7f4f0c013ff8
        t = 0x7f4f24032b18
        call = 0x7f4f2002afa8
        tt = 139977883331224
        events = 1
#7  0x00007f4f2eb4ca56 in triton_thread (thread=0x55883b94d5f8) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:192
        set = {__val = {516, 0 <repeats 15 times>}}
        sig = 10
        need_free = 0
        stack = 0x0
#8  0x00007f4f2eb1cfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139978033534720, -6510256976078385386, 140733014013950, 140733014013951, 139978033534720, 0, 6573199557991517974, 6573204482220674838}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#9  0x00007f4f2e6574cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 4 (Thread 0x7f4f2d072700 (LWP 27779)):
#0  0x00007f4f2e5964cc in __GI___sigtimedwait (set=set@entry=0x7f4f2d071d10, info=info@entry=0x7f4f2d071c40, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
        __arg4 = 8
        __arg2 = 139978034584640
        _a3 = 0
        _a1 = 139978034584848
        resultvar = <optimized out>
        __arg3 = 0
        __arg1 = 139978034584848
        _a4 = 8
        _a2 = 139978034584640
        resultvar = <optimized out>
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
        result = <optimized out>
        sc_ret = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        sc_cancel_oldtype = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  0x00007f4f2eb272bc in __sigwait (set=0x7f4f2d071d10, sig=0x7f4f2d071d00) at ../sysdeps/unix/sysv/linux/sigwait.c:28
        si = {si_signo = 1, si_errno = -1, si_code = 0, __pad0 = 0, _sifields = {_pad = {268581088, 32591, 268581016, 32591, 825110577, 775172398, 3618609, 32591, 0, 0, 201509912, 32591, 0, 0, 783631152, 32591, 783631152, 32591, 999293944, 21896, 755440864, 32591, 717405696, 2122344702, 755440864, 32591, 783597950, 32591}, _kill = {si_pid = 268581088, si_uid = 32591}, _timer = {si_tid = 268581088, si_overrun = 32591, si_sigval = {sival_int = 268581016, sival_ptr = 0x7f4f10023898}}, _rt = {si_pid = 268581088, si_uid = 32591, si_sigval = {sival_int = 268581016, sival_ptr = 0x7f4f10023898}}, _sigchld = {si_pid = 268581088, si_uid = 32591, si_status = 268581016, si_utime = 3329340098997006385, si_stime = 139977282762545}, _sigfault = {si_addr = 0x7f4f100238e0, si_addr_lsb = 14488, _bounds = {_addr_bnd = {_lower = 0x2e34312e312e3031, _upper = 0x7f4f00373731}, _pkey = 825110577}}, _sigpoll = {si_band = 139977547725024, si_fd = 268581016}, _sigsys = {_call_addr = 0x7f4f100238e0, _syscall = 268581016, _arch = 32591}}}
        ret = <optimized out>
#2  0x00007f4f2eb4c946 in triton_thread (thread=0x55883b9003e8) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:171
        set = {__val = {516, 0 <repeats 15 times>}}
        sig = 10
        need_free = 0
        stack = 0x0
#3  0x00007f4f2eb1cfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139978034587392, -6510256976078385386, 140733014013950, 140733014013951, 139978034587392, 0, 6573199145137786646, 6573204482220674838}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#4  0x00007f4f2e6574cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 3 (Thread 0x7f4f2d8f4700 (LWP 27778)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x558839ccbf68 <uc_cond+40>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
        __ret = -512
        oldtype = 0
        err = <optimized out>
        oldtype = <optimized out>
        err = <optimized out>
        __ret = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x558839ccbf00 <uc_lock>, cond=0x558839ccbf40 <uc_cond>) at pthread_cond_wait.c:502
        spin = 0
        buffer = {__routine = 0x7f4f2eb22d80 <__condvar_cleanup_waiting>, __arg = 0x7f4f2d8f3c90, __canceltype = 0, __prev = 0x0}
        cbuffer = {wseq = 0, cond = 0x558839ccbf40 <uc_cond>, mutex = 0x558839ccbf00 <uc_lock>, private = 0}
        rt = <optimized out>
        err = <optimized out>
        g = 0
        flags = <optimized out>
        g1_start = <optimized out>
        signals = <optimized out>
        result = 0
        wseq = 0
        seq = 0
        private = 0
        maxspin = <optimized out>
        err = <optimized out>
        result = <optimized out>
        wseq = <optimized out>
        g = <optimized out>
        seq = <optimized out>
        flags = <optimized out>
        private = <optimized out>
        signals = <optimized out>
        g1_start = <optimized out>
        spin = <optimized out>
        buffer = <optimized out>
        cbuffer = <optimized out>
        rt = <optimized out>
        s = <optimized out>
#2  __pthread_cond_wait (cond=0x558839ccbf40 <uc_cond>, mutex=0x558839ccbf00 <uc_lock>) at pthread_cond_wait.c:655
No locals.
#3  0x0000558839c9753a in uc_thread (unused=0x0) at /usr/local/src/accel-ppp/accel-pppd/ppp/ppp.c:350
        uc = 0x0
        fd = 0
        set = {__val = {18446744067266838271, 18446744073709551615 <repeats 15 times>}}
#4  0x00007f4f2eb1cfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139978043508480, -6510256976078385386, 140733014014046, 140733014014047, 139978043508480, 0, 6573198113271893782, 6573204482220674838}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#5  0x00007f4f2e6574cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 2 (Thread 0x7f4f2e0f5700 (LWP 27777)):
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
        set = {__val = {18446744067266838239, 1099511627520, 0 <repeats 14 times>}}
        pid = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
#1  0x00007f4f2e580535 in __GI_abort () at abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0}, sa_mask = {__val = {0 <repeats 13 times>, 139978051882256, 139977480731864, 139977480731808}}, sa_flags = 717405696, sa_restorer = 0x0}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x0000558839c91e3c in md_free (ptr=0x7f4f0c0212b8, fname=0x7f4f2e539060 "/usr/local/src/accel-ppp/accel-pppd/logs/log_file.c", line=156) at /usr/local/src/accel-ppp/accel-pppd/memdebug.c:84
        mem = 0x7f4f0c021280
#3  0x00007f4f2e536a92 in log_thread (unused=0x0) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:156
        lf = 0x7f4f0c0212d0
        iov = {{iov_base = 0x7f4f0c003c0c, iov_len = 39}, {iov_base = 0x7f4f0c024ecc, iov_len = 59}, {iov_base = 0x7f4f2004cecc, iov_len = 8}, {iov_base = 0x7f4f20026a0c, iov_len = 30}, {iov_base = 0x7f4f2005562c, iov_len = 61}, {iov_base = 0x7f4f2005562c, iov_len = 68}, {iov_base = 0x7f4f200223ac, iov_len = 32}, {iov_base = 0x7f4f2003520c, iov_len = 41}, {iov_base = 0x7f4f2000f64c, iov_len = 32}, {iov_base = 0x7f4f2003631c, iov_len = 52}, {iov_base = 0x7f4f2000bc8c, iov_len = 32}, {iov_base = 0x7f4f2001927c, iov_len = 25}, {iov_base = 0x7f4f0c029a5c, iov_len = 32}, {iov_base = 0x7f4f0c0219fc, iov_len = 70}, {iov_base = 0x7f4f0c007f0c, iov_len = 32}, {iov_base = 0x7f4f0c024ecc, iov_len = 70}, {iov_base = 0x7f4f0c03111c, iov_len = 32}, {iov_base = 0x7f4f0c02804c, iov_len = 66}, {iov_base = 0x7f4f0c0210dc, iov_len = 32}, {iov_base = 0x7f4f0c032cac, iov_len = 128}, {iov_base = 0x7f4f0c01952c, iov_len = 17}, {iov_base = 0x7f4f0c02349c, iov_len = 32}, {iov_base = 0x7f4f0c0352ec, iov_len = 128}, {iov_base = 0x7f4f0c01730c, iov_len = 128}, {iov_base = 0x7f4f0c033a3c, iov_len = 128}, {iov_base = 0x7f4f0c00f49c, iov_len = 35}, {iov_base = 0x7f4f0c040dcc, iov_len = 32}, {iov_base = 0x7f4f0c00d0dc, iov_len = 128}, {iov_base = 0x7f4f0c00ee0c, iov_len = 128}, {iov_base = 0x7f4f0c03fedc, iov_len = 128}, {iov_base = 0x7f4f0c03604c, iov_len = 128}, {iov_base = 0x7f4f0c031ccc, iov_len = 107}, {iov_base = 0x7f4f0c00bcbc, iov_len = 37}, {iov_base = 0x7f4f0c03d64c, iov_len = 76}, {iov_base = 0x7f4f0c01dc3c, iov_len = 37}, {iov_base = 0x7f4f0c034e0c, iov_len = 102}, {iov_base = 0x7f4f0c01648c, iov_len = 37}, {iov_base = 0x7f4f0c01e00c, iov_len = 47}, {iov_base = 0x7f4f0c04023c, iov_len = 37}, {iov_base = 0x7f4f0c021c3c, iov_len = 43}, {iov_base = 0x7f4f2002202c, iov_len = 36}, {iov_base = 0x7f4f20021eac, iov_len = 47}, {iov_base = 0x7f4f20030ebc, iov_len = 36}, {iov_base = 0x7f4f20030d3c, iov_len = 43}, {iov_base = 0x7f4f1002d5dc, iov_len = 37}, {iov_base = 0x7f4f1002d45c, iov_len = 42}, {iov_base = 0x7f4f2004ff3c, iov_len = 32}, {iov_base = 0x7f4f2002f26c, iov_len = 18}, {iov_base = 0x7f4f1002d71c, iov_len = 37}, {iov_base = 0x7f4f1002d59c, iov_len = 47}, {iov_base = 0x7f4f1002dc2c, iov_len = 37}, {iov_base = 0x7f4f1002daac, iov_len = 42}, {iov_base = 0x0, iov_len = 0} <repeats 877 times>, {iov_base = 0x0, iov_len = 139978062825196}, {iov_base = 0x0, iov_len = 1042}, {iov_base = 0x7f4f2e724500, iov_len = 1}, {iov_base = 0x7f4f2e560b1c, iov_len = 139978062826228}, {iov_base = 0x412, iov_len = 139978056596640}, {iov_base = 0x7f4f2e724500, iov_len = 139978051897480}, {iov_base = 0x7f4f2e0f4884, iov_len = 139978062825566}, {iov_base = 0x0, iov_len = 139978056596640}, {iov_base = 0x7f4f2e561fa8, iov_len = 139978062529867}, {iov_base = 0xa8428197, iov_len = 139978062521432}, {iov_base = 0x2a10a06, iov_len = 139978051897680}, {iov_base = 0x7f4f2e0f4940, iov_len = 139978051897480}, {iov_base = 0x55883b905d70, iov_len = 25}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 2822930839}, {iov_base = 0x7f4f2eb55d80, iov_len = 139978051897832}, {iov_base = 0x7f4f2eb55a20, iov_len = 0}, {iov_base = 0x7f4f2e7251d0, iov_len = 139978062828431}, {iov_base = 0x8, iov_len = 139978058387920}, {iov_base = 0x5, iov_len = 0}, {iov_base = 0x1, iov_len = 139978062780960}, {iov_base = 0x7f4f2eb1854b, iov_len = 4294967296}, {iov_base = 0x7f4f2e0f4940, iov_len = 139978051897680}, {iov_base = 0x7f4f2eb55d80, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0xffffffff, iov_len = 0}, {iov_base = 0x7f4f2e568158, iov_len = 139978058384640}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 139978062631232}, {iov_base = 0x0, iov_len = 140733014014046}, {iov_base = 0x7ffef54ef85f, iov_len = 139978051901184}, {iov_base = 0x0, iov_len = 139978062846720}, {iov_base = 0x5, iov_len = 0}, {iov_base = 0x0, iov_len = 139978056565080}, {iov_base = 0x7f4f2e0f4da0, iov_len = 139978062873786}, {iov_base = 0x7f4f2e0f4e48, iov_len = 139978062548777}, {iov_base = 0x7f4f2e0f4dd0, iov_len = 24}, {iov_base = 0x7f4f2e0f4dd0, iov_len = 139978051901184}, {iov_base = 0x7f4f2e0f5700, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 281470681751424}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x7f4f2e71a560 <_nl_global_locale>, iov_len = 139978058356992}, {iov_base = 0x1000, iov_len = 0}, {iov_base = 0x800000, iov_len = 0}, {iov_base = 0xff0000, iov_len = 4278190080}, {iov_base = 0xa090a090a090a09, iov_len = 723120249109023241}, {iov_base = 0xa090a090a090a09, iov_len = 723120249109023241}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0xa090a090a090a09, iov_len = 723120249109023241}, {iov_base = 0xffffffff, iov_len = 1099511627520}, {iov_base = 0x0, iov_len = 0} <repeats 12 times>, {iov_base = 0x2, iov_len = 0}, {iov_base = 0x0, iov_len = 0} <repeats 21 times>}
        chunk = 0x7f4f0c03bce0
        msg = 0x7f4f0c03fcd8
        iov_cnt = 2
        msg_list = {next = 0x7f4f2e0f0d00, prev = 0x7f4f2e0f0d00}
        free_list = {next = 0x7f4f2e0f0d10, prev = 0x7f4f2e0f0d10}
        set = {__val = {18446744067266838271, 18446744073709551615 <repeats 15 times>}}
#4  0x00007f4f2eb1cfa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139978051901184, -6510256976078385386, 140733014014046, 140733014014047, 139978051901184, 0, 6573205809316417302, 6573204482220674838}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#5  0x00007f4f2e6574cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 1 (Thread 0x7f4f2e55ccc0 (LWP 27776)):
#0  0x00007f4f2e5964cc in __GI___sigtimedwait (set=set@entry=0x7ffef54ef970, info=info@entry=0x7ffef54ef880, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
        __arg4 = 8
        __arg2 = 140733014014080
        _a3 = 0
        _a1 = 140733014014320
        resultvar = <optimized out>
        __arg3 = 0
        __arg1 = 140733014014320
        _a4 = 8
        _a2 = 140733014014080
        resultvar = <optimized out>
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
        result = <optimized out>
        sc_ret = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        sc_cancel_oldtype = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  0x00007f4f2eb272bc in __sigwait (set=0x7ffef54ef970, sig=0x7ffef54ef944) at ../sysdeps/unix/sysv/linux/sigwait.c:28
        si = {si_signo = 0, si_errno = 0, si_code = 0, __pad0 = 0, _sifields = {_pad = {0 <repeats 28 times>}, _kill = {si_pid = 0, si_uid = 0}, _timer = {si_tid = 0, si_overrun = 0, si_sigval = {sival_int = 0, sival_ptr = 0x0}}, _rt = {si_pid = 0, si_uid = 0, si_sigval = {sival_int = 0, sival_ptr = 0x0}}, _sigchld = {si_pid = 0, si_uid = 0, si_status = 0, si_utime = 0, si_stime = 0}, _sigfault = {si_addr = 0x0, si_addr_lsb = 0, _bounds = {_addr_bnd = {_lower = 0x0, _upper = 0x0}, _pkey = 0}}, _sigpoll = {si_band = 0, si_fd = 0}, _sigsys = {_call_addr = 0x0, _syscall = 0, _arch = 0}}}
        ret = <optimized out>
#2  0x0000558839cbceeb in main (_argc=5, _argv=0x7ffef54efb78) at /usr/local/src/accel-ppp/accel-pppd/main.c:389
        set = {__val = {16386, 0 <repeats 15 times>}}
        i = 5
        sig = 21896
        goto_daemon = 0
        len = 32766
        pid = 0
        sa = {__sigaction_handler = {sa_handler = 0x558839cbc2d9 <sigsegv>, sa_sigaction = 0x558839cbc2d9 <sigsegv>}, sa_mask = {__val = {18446744067267100671, 18446744073709551615 <repeats 15 times>}}, sa_flags = 0, sa_restorer = 0x0}
        pagesize = 4096
        internal = 0
        no_sigint = 0
        no_sigsegv = 0
Detaching from program: /usr/sbin/accel-pppd, process 27776
[Inferior 1 (process 27776) detached]
2020-07-12-03-33-04-679928187 accelppp crashed/stopped now
last lines from debug.log

Code: Select all

[2020-07-12 03:32:59.550] l2tp tunnel 26405-15573 (masked_ipv4_2:1701): new session 40399-10007 created following reception of ICRQ
[2020-07-12 03:32:59.550] l2tp tunnel 26405-15573 (masked_ipv4_2:1701): send [L2TP tid=15573 sid=10007 <Message-Type Incoming-Call-Reply> <Assigned-Session-ID -25137>]
[2020-07-12 03:32:59.556] l2tp tunnel 26405-15573 (masked_ipv4_2:1701): recv [L2TP tid=26405 sid=40399 <Message-Type Incoming-Call-Connected> <TX-Speed 0> <Framing-Type 1> <RX-Speed 0>]
[2020-07-12 03:32:59.556] l2tp session 26405-15573, 40399-10007: handling ICCN
[2020-07-12 03:32:59.556] : : starting data channel for l2tp(masked_ipv4_2:1701 session 26405-15573, 40399-10007)
[2020-07-12 03:32:59.556] : ddbe02d5f0f68ddf: send [LCP ConfReq id=94 <auth MSCHAP-v2> <mru 1450> <magic 38306e7a>]
[2020-07-12 03:32:59.572] l2tp tunnel 23938-37005 (masked_ipv4_1:52831): retransmission #1
[2020-07-12 03:32:59.572] l2tp tunnel 23938-37005 (masked_ipv4_1:52831): retransmit (timeout) [L2TP tid=37005 sid=0 <Message-Type Hello>]
[2020-07-12 03:32:59.585] : ddbe02d5f0f68ddf: recv [LCP ConfReq id=1 <mru 1420> < 2 6 0 0 0 0 > <magic c027f8c3>]
[2020-07-12 03:32:59.586] : ddbe02d5f0f68ddf: send [LCP ConfRej id=1  < 2 6 0 0 0 0 >]
[2020-07-12 03:32:59.599] : ddbe02d5f0f68ddf: recv [LCP ConfReq id=2 <mru 1420> <magic c027f8c3>]
[2020-07-12 03:32:59.599] : ddbe02d5f0f68ddf: send [LCP ConfAck id=2 ]
[2020-07-12 03:33:02.557] : ddbe02d5f0f68ddf: send [LCP ConfReq id=94 <auth MSCHAP-v2> <mru 1450> <magic 38306e7a>]
[2020-07-12 03:33:02.562] : ddbe02d5f0f68ddf: recv [LCP ConfAck id=94 <auth MSCHAP-v2> <mru 1450> <magic 38306e7a>]
[2020-07-12 03:33:02.562] : ddbe02d5f0f68ddf: send [MSCHAP-v2 Challenge id=1 <5684deaa69246659dcaa728624321a0>]
[2020-07-12 03:33:02.568] : ddbe02d5f0f68ddf: recv [MSCHAP-v2 Response id=1 <6c2fa071684ef0d19daa2d2c50e4c829>, <ae89583d7ca3d2dd7d066b44d97eb15a4115ad1983a52>, F=0, name="masked_l2tp_username"]
[2020-07-12 03:33:02.568] : ddbe02d5f0f68ddf: send [RADIUS(1) Access-Request id=1 <User-Name "masked_l2tp_username"> <NAS-Identifier "accel-ppp"> <NAS-Port-Type Virtual> <Service-Type Framed-User> <Framed-Protocol PPP> <Calling-Station-Id "masked_ipv4_2"> <Called-Station-Id "masked_our_bng_ip"> <MS-CHAP-Challenge masked-chap-thing]
[2020-07-12 03:33:02.570] : ddbe02d5f0f68ddf: recv [RADIUS(1) Access-Accept id=1 <Framed-Protocol PPP> <Framed-Compression Van-Jacobson-TCP-IP> <Framed-IP-Address masked_subs_ip_1> <NAS-Port-Id "masked_subs_id"> <Delegated-IPv6-Prefix 2a0f:de00:fe00:4400::/56> <MS-Primary-DNS-Server 1.1.1.1> <MS-Secondary-DNS-Server 1.0.0.1> masked-chap-thing-2]
[2020-07-12 03:33:02.570] ppp24: ddbe02d5f0f68ddf: connect: ppp24 <--> l2tp(masked_ipv4_2:1701 session 26405-15573, 40399-10007)
[2020-07-12 03:33:02.570] ppp24: ddbe02d5f0f68ddf: send [MSCHAP-v2 Success id=1 "S=BA1FC0D07F35C76F5D3108A30DA9EC1A75ECC1C0 M=Authentication succeeded"]
[2020-07-12 03:33:02.571] ppp24: ddbe02d5f0f68ddf: send [IPCP ConfReq id=48 <addr 192.168.155.1>]
[2020-07-12 03:33:02.571] ppp24: ddbe02d5f0f68ddf: masked_l2tp_username: authentication succeeded
[2020-07-12 03:33:02.586] ppp24: ddbe02d5f0f68ddf: recv [IPCP ConfReq id=1 <addr 0.0.0.0> <dns1 0.0.0.0> <dns2 0.0.0.0>]
[2020-07-12 03:33:02.586] ppp24: ddbe02d5f0f68ddf: send [IPCP ConfNak id=1 <addr masked_subs_ip_1> <dns1 1.1.1.1> <dns2 1.0.0.1>]
[2020-07-12 03:33:02.588] ppp24: ddbe02d5f0f68ddf: recv [IPV6CP ConfReq id=1 <addr masked1>]
[2020-07-12 03:33:02.588] ppp24: ddbe02d5f0f68ddf: send [IPV6CP ConfReq id=b8 <addr masked2>]
[2020-07-12 03:33:02.588] ppp24: ddbe02d5f0f68ddf: send [IPV6CP ConfAck id=1]
[2020-07-12 03:33:02.588] ppp24: ddbe02d5f0f68ddf: recv [IPCP ConfAck id=48 <addr 192.168.155.1>]
[2020-07-12 03:33:02.599] ppp24: ddbe02d5f0f68ddf: recv [IPCP ConfReq id=2 <addr masked_subs_ip_1> <dns1 1.1.1.1> <dns2 1.0.0.1>]
[2020-07-12 03:33:02.599] ppp24: ddbe02d5f0f68ddf: send [IPCP ConfAck id=2]
[2020-07-12 03:33:02.600] ppp24: ddbe02d5f0f68ddf: recv [IPV6CP ConfAck id=b8 <addr masked2>]
[2020-07-12 03:33:02.623] ppp24: ddbe02d5f0f68ddf: rename interface to 'masked_subs_id'
[2020-07-12 03:33:02.623] l2tp tunnel 26405-15573 (masked_ipv4_2:1701): recv [L2TP tid=26405 sid=0 <Message-Type Stop-Ctrl-Conn-Notify> <Assigned-Tunnel-ID 15573> <Result-Code>]
[2020-07-12 03:33:02.623] l2tp tunnel 26405-15573 (masked_ipv4_2:1701): handling StopCCN
[2020-07-12 03:33:02.623] l2tp tunnel 26405-15573 (masked_ipv4_2:1701): StopCCN received from peer (result: 1, error: 0, message: "Goodbye!"), disconnecting tunnel
[2020-07-12 03:33:02.623] l2tp session 26405-15573, 40399-10007: deleting session
[2020-07-12 03:33:02.623] masked_subs_id: ddbe02d5f0f68ddf: session started over l2tp session 26405-15573, 40399-10007
[2020-07-12 03:33:02.623] l2tp session 26405-15573, 40399-10007: deleting data channel
[2020-07-12 03:33:02.623] masked_subs_id: ddbe02d5f0f68ddf: failed to rename ppp to default name
[2020-07-12 03:33:05.043] accel-ppp version b9777b182688240e7c14dfb0a0a97f4a0039533e
From another crash (Error2)

syslog looks like this

Code: Select all

Jul 17 06:25:12 bng systemd[10150]: Reached target Shutdown.
Jul 17 06:25:12 bng systemd[10150]: systemd-exit.service: Succeeded.
Jul 17 06:25:12 bng systemd[10150]: Started Exit the Session.
Jul 17 06:25:12 bng systemd[10150]: Reached target Exit the Session.
Jul 17 06:25:12 bng systemd[1]: user@65534.service: Succeeded.
Jul 17 06:25:12 bng systemd[1]: Stopped User Manager for UID 65534.
Jul 17 06:25:12 bng systemd[1]: Stopping User Runtime Directory /run/user/65534...
Jul 17 06:25:12 bng systemd[1]: run-user-65534.mount: Succeeded.
Jul 17 06:25:12 bng systemd[5587]: run-user-65534.mount: Succeeded.
Jul 17 06:25:12 bng systemd[1]: user-runtime-dir@65534.service: Succeeded.
Jul 17 06:25:12 bng systemd[1]: Stopped User Runtime Directory /run/user/65534.
Jul 17 06:25:12 bng systemd[1]: Removed slice User Slice of UID 65534.
Jul 17 06:37:32 bng systemd-udevd[10181]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jul 17 06:37:32 bng kernel: [350743.881814] i0066c0: renamed from ppp47
Jul 17 06:48:22 bng systemd[1]: Starting Daily apt upgrade and clean activities...
Jul 17 06:48:23 bng systemd[1]: apt-daily-upgrade.service: Succeeded.
Jul 17 06:48:23 bng systemd[1]: Started Daily apt upgrade and clean activities.
Jul 17 07:17:01 bng CRON[10248]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jul 17 07:31:13 bng systemd[1]: accel-ppp.service: Main process exited, code=killed, status=6/ABRT
Jul 17 07:31:13 bng systemd[1]: accel-ppp.service: Failed with result 'signal'.
Jul 17 07:31:13 bng systemd[1]: accel-ppp.service: Service RestartSec=100ms expired, scheduling restart.
Jul 17 07:31:13 bng systemd[1]: accel-ppp.service: Scheduled restart job, restart counter is at 1.
Jul 17 07:31:13 bng systemd[1]: Stopped Accel-PPP.
Jul 17 07:31:13 bng systemd[1]: Started Accel-PPP.
debug.log

Code: Select all

[2020-07-17 07:30:34.320] l2tp tunnel 11131-64464 (maskedip3:12920): retransmit (timeout) [L2TP tid=64464 sid=0 <Message-Type Hello>]
[2020-07-17 07:30:42.660] l2tp tunnel 8973-52994 (maskedip1:62192): retransmission #1
[2020-07-17 07:30:42.660] l2tp tunnel 8973-52994 (maskedip1:62192): retransmit (timeout) [L2TP tid=52994 sid=0 <Message-Type Hello>]
[2020-07-17 07:30:43.460] l2tp tunnel 48869-4746 (maskedip2:1417): retransmission #1
[2020-07-17 07:30:43.460] l2tp tunnel 48869-4746 (maskedip2:1417): retransmit (timeout) [L2TP tid=4746 sid=0 <Message-Type Hello>]
[2020-07-17 07:30:44.325] l2tp tunnel 11131-64464 (maskedip3:12920): retransmission #1
[2020-07-17 07:30:44.325] l2tp tunnel 11131-64464 (maskedip3:12920): retransmit (timeout) [L2TP tid=64464 sid=0 <Message-Type Hello>]
[2020-07-17 07:30:52.656] l2tp tunnel 8973-52994 (maskedip1:62192): retransmission #1
[2020-07-17 07:30:52.656] l2tp tunnel 8973-52994 (maskedip1:62192): retransmit (timeout) [L2TP tid=52994 sid=0 <Message-Type Hello>]
[2020-07-17 07:30:53.459] l2tp tunnel 48869-4746 (maskedip2:1417): retransmission #1
[2020-07-17 07:30:53.459] l2tp tunnel 48869-4746 (maskedip2:1417): retransmit (timeout) [L2TP tid=4746 sid=0 <Message-Type Hello>]
[2020-07-17 07:30:54.342] l2tp tunnel 11131-64464 (maskedip3:12920): retransmission #1
[2020-07-17 07:30:54.342] l2tp tunnel 11131-64464 (maskedip3:12920): retransmit (timeout) [L2TP tid=64464 sid=0 <Message-Type Hello>]
[2020-07-17 07:31:02.662] l2tp tunnel 8973-52994 (maskedip1:62192): retransmission #1
[2020-07-17 07:31:02.662] l2tp tunnel 8973-52994 (maskedip1:62192): retransmit (timeout) [L2TP tid=52994 sid=0 <Message-Type Hello>]
[2020-07-17 07:31:03.460] l2tp tunnel 48869-4746 (maskedip2:1417): retransmission #1
[2020-07-17 07:31:03.461] l2tp tunnel 48869-4746 (maskedip2:1417): retransmit (timeout) [L2TP tid=4746 sid=0 <Message-Type Hello>]
[2020-07-17 07:31:04.319] l2tp tunnel 11131-64464 (maskedip3:12920): retransmission #1
[2020-07-17 07:31:04.319] l2tp tunnel 11131-64464 (maskedip3:12920): retransmit (timeout) [L2TP tid=64464 sid=0 <Message-Type Hello>]
[2020-07-17 07:31:07.268] maskedsubid1: ddbe02d5f0f69672: recv [DHCPv6 Solicit XID=644866 <Client-ID 3:0001d4ca6de71a00> <Elapsed-Time 234918954> <Rapid-Commit> <IA-PD 6 T1=
1800 T2=2880>]
[2020-07-17 07:31:07.268] maskedsubid1: ddbe02d5f0f69672: send [DHCPv6 Reply XID=644866 <Server-ID 3:001b0000000000000001> <Client-ID 3:0001d4ca6de71a00> <Rapid-Commit> <IA-
PD 6 T1=302400 T2=483840 {IA-Prefix masked_ipv6_1/56 pref_lifetime=604800 valid_lifetime=2592000}> <Preference 255>]
[2020-07-17 07:31:11.419] l2tp tunnel 30011-2821 (maskedip4:1701): recv [L2TP tid=30011 sid=0 <Message-Type Stop-Ctrl-Conn-Notify> <Assigned-Tunnel-ID 2821> <Result-Co
de>]
[2020-07-17 07:31:11.419] l2tp tunnel 30011-2821 (maskedip4:1701): handling StopCCN
[2020-07-17 07:31:11.419] l2tp tunnel 30011-2821 (maskedip4:1701): StopCCN received from peer (result: 1, error: 0, message: "Goodbye!"), disconnecting tunnel
[2020-07-17 07:31:11.419] maskedsubid2: ddbe02d5f0f698d3: recv [LCP TermReq id=3]
[2020-07-17 07:31:11.419] l2tp session 30011-2821, 32733-50573: deleting session
[2020-07-17 07:31:11.419] maskedsubid2: ddbe02d5f0f698d3: send [LCP TermAck id=3]
[2020-07-17 07:31:11.419] l2tp session 30011-2821, 32733-50573: deleting data channel
[2020-07-17 07:31:13.664] accel-ppp version b9777b182688240e7c14dfb0a0a97f4a0039533e
exopedro
Posts: 8
Joined: 23 Jun 2020, 15:17

Re: accelppp exits unexpectedly with log error md:epoll_wait

Post by exopedro »

Hi again,

We finally adjusted appropriately the systemd to get the coredump file for the SIGABRT signal [1]. And now we received a new SIGABRT from debian's systemd, so we can share privately with you the entire coredump and the related binary file accel-pppd.

We also received the regular exit -1 (255) error (as pointed in previous entries) but this time it was captured with debug symbols and gdb attach. Our new gdb attach script [2] has its coredump (and full backtrace of all threads).

We are only going to detail here the latest error (which is cause why we opened this forum post).

According to the backtrace of the coredump there is a segmentation fault in log_file.c line 269 [3] (which is the same line in our code, stable and trunk). This happens after calling per-user logging functions, and it comes after destroying an l2tp session connection attempt from a user previously disabled in radius.

The struct of the failed l2tp session is displayed on the coredump output

coredump output

Code: Select all

[12:45:53] root@bng:/media/dlogs# gdb accel-pppd coredump-script-2020-07-31-23-36-20-839756840
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from accel-pppd...done.
[New LWP 29183]
[New LWP 29179]
[New LWP 29180]
[New LWP 29181]
[New LWP 29182]
[New LWP 29184]
[New LWP 29185]
[New LWP 29186]
[New LWP 29187]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/accel-pppd'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f3e3bdbd0f3 in find_pd (ses=0x7f3e34042ee8, pd_key=0x7f3e3bdc12e8 <pd_key1>) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:269
269             list_for_each_entry(pd, &ses->pd_list, entry) {
[Current thread is 1 (Thread 0x7f3e3a7f7700 (LWP 29183))]
(gdb) bt
#0  0x00007f3e3bdbd0f3 in find_pd (ses=0x7f3e34042ee8, pd_key=0x7f3e3bdc12e8 <pd_key1>) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:269
#1  0x00007f3e3bdbd15f in find_lpd (ses=0x7f3e34042ee8, pd_key=0x7f3e3bdc12e8 <pd_key1>) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:280
#2  0x00007f3e3bdbd246 in per_user_log (t=0x7f3e3bdc1240 <per_user_target>, msg=0x7f3e1c0133a8, ses=0x7f3e34042ee8) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:308
#3  0x0000561e48d03839 in do_log (level=4, fmt=0x7f3e3bdaf3d1 "session destroyed\n", ap=0x7f3e3a7f6b90, ses=0x7f3e34042ee8) at /usr/local/src/accel-ppp/accel-pppd/log.c:109
#4  0x0000561e48d0425e in log_ppp_info2 (fmt=0x7f3e3bdaf3d1 "session destroyed\n") at /usr/local/src/accel-ppp/accel-pppd/log.c:220
#5  0x00007f3e3bd9d13a in __apses_destroy (data=0x7f3e34042de8) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:1246
#6  0x00007f3e3c3d2d6b in ctx_thread (ctx=0x7f3e1c00b788) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:272
#7  0x00007f3e3c3d2a56 in triton_thread (thread=0x561e4a88e208) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:192
#8  0x00007f3e3c3a2fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#9  0x00007f3e3bedd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) select-frame 5
(gdb) print sess
$1 = (struct l2tp_sess_t *) 0x7f3e34042de8
(gdb) print sess.ppp
$2 = {ses = {entry = {next = 0x0, prev = 0x0}, state = 3, chan_name = 0x7f3e1c0316c8 "masked_ip3:1701 session 8104-39532, 51086-858", ifname = '\000' <repeats 15 times>,
    ifname_rename = 0x0, unit_idx = -1, ifindex = -1, sessionid = "ddbe02d5f0f74d18", start_time = 1104163, stop_time = 1104164, idle_time = 1104163, username = 0x0, ipv4 = 0x0,
    ipv6 = 0x0, ipv6_dp = 0x0, ipv4_pool_name = 0x0, ipv6_pool_name = 0x0, net = 0x561e4a841ea8, ctrl = 0x7f3e34042e90, comp = 0x0, wakeup = 0x0, terminating = -1,
    terminated = -1, down = -1, terminate_cause = 1, pd_list = {next = 0x7f3e1c024678, prev = 0x7f3e1c025068}, idle_timeout = 0, session_timeout = 0, timer = {tpd = 0x0,
      expire_tv = {tv_sec = 0, tv_usec = 0}, period = 0, expire = 0x0}, acct_rx_bytes = 0, acct_tx_bytes = 0, acct_input_gigawords = 0, acct_output_gigawords = 0,
    acct_rx_packets_i = 0, acct_tx_packets_i = 0, acct_rx_bytes_i = 0, acct_tx_bytes_i = 0, acct_start = 0}, chan_hnd = {tpd = 0x0, fd = -1,
    read = 0x561e48ce0667 <ppp_chan_read>, write = 0x0}, unit_hnd = {tpd = 0x0, fd = 0, read = 0x0, write = 0x0}, fd = -1, chan_fd = -1, unit_fd = -1, chan_idx = 38920,
  mtu = 1420, mru = 1500, buf = 0x0, buf_size = -1, chan_handlers = {next = 0x7f3e34043080, prev = 0x7f3e34043080}, unit_handlers = {next = 0x7f3e34043090,
    prev = 0x7f3e34043090}, layers = {next = 0x7f3e340430a0, prev = 0x7f3e340430a0}}
gdb with thread apply all bt full

Code: Select all

================== 2020-07-31-23-36-20-837622330 ===================
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 29179
[New LWP 29180]
[New LWP 29181]
[New LWP 29182]
[New LWP 29183]
[New LWP 29184]
[New LWP 29185]
[New LWP 29186]
[New LWP 29187]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007f3e3be1c4cc in __GI___sigtimedwait (set=set@entry=0x7ffcdbeaee10, info=info@entry=0x7ffcdbeaed20, timeout=timeout@entry=0x0)
    at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
29	../sysdeps/unix/sysv/linux/sigtimedwait.c: No such file or directory.
Breakpoint 1 at 0x7f3e3beaa9a0: file ../sysdeps/unix/sysv/linux/_exit.c, line 27.
Continuing.

Thread 5 "accel-pppd" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f3e3a7f7700 (LWP 29183)]
0x00007f3e3bdbd0f3 in find_pd (ses=0x7f3e34042ee8, pd_key=0x7f3e3bdc12e8 <pd_key1>) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:269
269		list_for_each_entry(pd, &ses->pd_list, entry) {

Thread 9 (Thread 0x7f3e39cf3700 (LWP 29187)):
#0  0x00007f3e3bedd7ef in epoll_wait (epfd=9, events=0x561e4a811748, maxevents=64, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
        resultvar = 18446744073709551612
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  0x00007f3e3c3d193e in timer_thread (arg=0x0) at /usr/local/src/accel-ppp/accel-pppd/triton/timer.c:82
        i = 1
        n = 1
        r = 1
        t = 0x7f3e3001de48
        set = {__val = {18446744067266838271, 18446744073709551615 <repeats 15 times>}}
#2  0x00007f3e3c3a2fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139905234581248, -7190569452712916160, 140723998092510, 140723998092511, 139905234581248, 0, 7082337781770718016, 7082349506948729664}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#3  0x00007f3e3bedd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 8 (Thread 0x7f3e3a4f4700 (LWP 29186)):
#0  0x00007f3e3bedd7ef in epoll_wait (epfd=8, events=0x561e4a8110c8, maxevents=64, timeout=-1) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
        resultvar = 18446744073709551612
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
#1  0x00007f3e3c3d0b6d in md_thread (arg=0x0) at /usr/local/src/accel-ppp/accel-pppd/triton/md.c:75
        i = 1
        n = 1
        r = 1
        h = 0x7f3e30040798
        set = {__val = {18446744067266838271, 18446744073709551615 <repeats 15 times>}}
#2  0x00007f3e3c3a2fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139905242973952, -7190569452712916160, 140723998092510, 140723998092511, 139905242973952, 0, 7082336682795961152, 7082349506948729664}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#3  0x00007f3e3bedd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 7 (Thread 0x7f3e3a5f5700 (LWP 29185)):
#0  0x00007f3e3be1c4cc in __GI___sigtimedwait (set=set@entry=0x7f3e3a5f4d10, info=info@entry=0x7f3e3a5f4c40, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
        __arg4 = 8
        __arg2 = 139905244023872
        _a3 = 0
        _a1 = 139905244024080
        resultvar = <optimized out>
        __arg3 = 0
        __arg1 = 139905244024080
        _a4 = 8
        _a2 = 139905244023872
        resultvar = <optimized out>
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
        result = <optimized out>
        sc_ret = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        sc_cancel_oldtype = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  0x00007f3e3c3ad2bc in __sigwait (set=0x7f3e3a5f4d10, sig=0x7f3e3a5f4d00) at ../sysdeps/unix/sysv/linux/sigwait.c:28
        si = {si_signo = 0, si_errno = -1, si_code = 0, __pad0 = 0, _sifields = {_pad = {872632432, 32574, 872632360, 32574, 825110577, 875444526, 1010630708, 32574, 0, 0, 805559352, 32574, 0, 0, 1010672432, 32574, 1010672432, 32574, 1250485880, 22046, 979324128, 32574, -655010560, 579136882, 979324128, 32574, 1010639230, 32574}, _kill = {si_pid = 872632432, si_uid = 32574}, _timer = {si_tid = 872632432, si_overrun = 32574, si_sigval = {sival_int = 872632360, sival_ptr = 0x7f3e34035028}}, _rt = {si_pid = 872632432, si_uid = 32574, si_sigval = {sival_int = 872632360, sival_ptr = 0x7f3e34035028}}, _sigchld = {si_pid = 872632432, si_uid = 32574, si_status = 872632360, si_utime = 3760005609457332273, si_stime = 139905275330612}, _sigfault = {si_addr = 0x7f3e34035070, si_addr_lsb = 20520, _bounds = {_addr_bnd = {_lower = 0x342e392e312e3031, _upper = 0x7f3e3c3d0034 <abort@plt+4>}, _pkey = 825110577}}, _sigpoll = {si_band = 139905137332336, si_fd = 872632360}, _sigsys = {_call_addr = 0x7f3e34035070, _syscall = 872632360, _arch = 32574}}}
        ret = <optimized out>
#2  0x00007f3e3c3d2946 in triton_thread (thread=0x561e4a88e668) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:171
        set = {__val = {516, 0 <repeats 15 times>}}
        sig = 10
        need_free = 0
        stack = 0x0
#3  0x00007f3e3c3a2fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139905244026624, -7190569452712916160, 140723998092446, 140723998092447, 139905244026624, 0, 7082336545893878592, 7082349506948729664}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#4  0x00007f3e3bedd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 6 (Thread 0x7f3e3a6f6700 (LWP 29184)):
#0  0x00007f3e3bdbd0f3 in find_pd (ses=0x7f3e34042ee8, pd_key=0x7f3e3bdc12f0 <pd_key2>) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:269
        pd = 0x0
#1  0x00007f3e3bdbd15f in find_lpd (ses=0x7f3e34042ee8, pd_key=0x7f3e3bdc12f0 <pd_key2>) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:280
        pd = 0x7f3e1c025068
#2  0x00007f3e3bdbde46 in ev_ctrl_finished (ses=0x7f3e34042ee8) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:576
        lpd = 0x7f3e1c025068
        fpd = 0x7f3e30026428
        fname = 0x7f3e3001de48 ""
#3  0x00007f3e3c3d6b44 in triton_event_fire (ev_id=8, arg=0x7f3e34042ee8) at /usr/local/src/accel-ppp/accel-pppd/triton/event.c:103
        ev = 0x561e4a84aed0
        h = 0x561e4a84aef0
#4  0x00007f3e3bd9c6b9 in l2tp_session_free (sess=0x7f3e34042de8) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:1023
        pack = 0x7f3e3c3da2f0 <freed_list>
        cause = 6
        res = 1
#5  0x00007f3e3bed9354 in tdestroy_recurse (root=0x7f3e1c00ea20, freefct=0x7f3e3bd9c594 <l2tp_session_free>) at tsearch.c:733
No locals.
#6  0x00007f3e3bd9be32 in l2tp_tunnel_free_sessions (conn=0x7f3e1c0134a8) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:853
        sessions = 0x7f3e1c00ea20
#7  0x00007f3e3bda24af in l2tp_tunnel_finwait (conn=0x7f3e1c0134a8) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:2689
        rtimeout = 980376254
        indx = 3208
#8  0x00007f3e3bda4388 in l2tp_recv_StopCCN (conn=0x7f3e1c0134a8, pack=0x7f3e3004bb98) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:3210
        assigned_tid = 0x7f3e300262a8
        result_code = 0x7f3e1c035048
        attr = 0x7f3e3004bbd8
        err_msg = 0x7f3e3000a748 "Goodbye!"
        res = 1
        err = 0
#9  0x00007f3e3bda7488 in l2tp_tunnel_recv (conn=0x7f3e1c0134a8, pack=0x7f3e3004bb98, msg_type=4, mandatory=-1) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:4071
No locals.
#10 0x00007f3e3bda7ff8 in l2tp_tunnel_reply (conn=0x7f3e1c0134a8, need_ack=1) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:4272
        msg_attr = 0x7f3e3001adc8
        pack = 0x7f3e3004bb98
        sess = 0x2284ed72d8f55500
        msg_sid = 0
        msg_type = 4
        id = 4
        pkt_count = 1
        res = 0
#11 0x00007f3e3bda858c in l2tp_conn_read (h=0x7f3e1c0134f0) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:4367
        conn = 0x7f3e1c0134a8
        pack = 0x0
        pkt_count = 1
        need_ack = 1
        res = -1
#12 0x00007f3e3c3d2caa in ctx_thread (ctx=0x7f3e3000be88) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:251
        h = 0x7f3e30013a38
        t = 0x0
        call = 0x7f3e3bd99b00 <l2tp_ctx_switch>
        tt = 139904734541008
        events = 1
#13 0x00007f3e3c3d2a56 in triton_thread (thread=0x561e4a88e438) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:192
        set = {__val = {516, 0 <repeats 15 times>}}
        sig = 10
        need_free = 0
        stack = 0x0
#14 0x00007f3e3c3a2fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139905245079296, -7190569452712916160, 140723998092446, 140723998092447, 139905245079296, 0, 7082336958747609920, 7082349506948729664}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#15 0x00007f3e3bedd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 5 (Thread 0x7f3e3a7f7700 (LWP 29183)):
#0  0x00007f3e3bdbd0f3 in find_pd (ses=0x7f3e34042ee8, pd_key=0x7f3e3bdc12e8 <pd_key1>) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:269
        pd = 0x0
#1  0x00007f3e3bdbd15f in find_lpd (ses=0x7f3e34042ee8, pd_key=0x7f3e3bdc12e8 <pd_key1>) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:280
        pd = 0x1231c0133a8
#2  0x00007f3e3bdbd246 in per_user_log (t=0x7f3e3bdc1240 <per_user_target>, msg=0x7f3e1c0133a8, ses=0x7f3e34042ee8) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:308
        lpd = 0x7f3e1c0133a8
#3  0x0000561e48d03839 in do_log (level=4, fmt=0x7f3e3bdaf3d1 "session destroyed\n", ap=0x7f3e3a7f6b90, ses=0x7f3e34042ee8) at /usr/local/src/accel-ppp/accel-pppd/log.c:109
        t = 0x7f3e3bdc1240 <per_user_target>
        m = 0x7f3e1c0133a8
#4  0x0000561e48d0425e in log_ppp_info2 (fmt=0x7f3e3bdaf3d1 "session destroyed\n") at /usr/local/src/accel-ppp/accel-pppd/log.c:220
        ap = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7f3e3a7f6c70, reg_save_area = 0x7f3e3a7f6bb0}}
#5  0x00007f3e3bd9d13a in __apses_destroy (data=0x7f3e34042de8) at /usr/local/src/accel-ppp/accel-pppd/ctrl/l2tp/l2tp.c:1246
        sess = 0x7f3e34042de8
#6  0x00007f3e3c3d2d6b in ctx_thread (ctx=0x7f3e1c00b788) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:272
        h = 0x7f3e30040798
        t = 0x7f3e34042ee8
        call = 0x7f3e1c009ec8
        tt = 139905137389160
        events = 1
#7  0x00007f3e3c3d2a56 in triton_thread (thread=0x561e4a88e208) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:192
        set = {__val = {516, 0 <repeats 15 times>}}
        sig = 10
        need_free = 0
        stack = 0x0
#8  0x00007f3e3c3a2fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139905246131968, -7190569452712916160, 140723998092446, 140723998092447, 139905246131968, 0, 7082336821845527360, 7082349506948729664}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#9  0x00007f3e3bedd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 4 (Thread 0x7f3e3a8f8700 (LWP 29182)):
#0  0x00007f3e3be1c4cc in __GI___sigtimedwait (set=set@entry=0x7f3e3a8f7d10, info=info@entry=0x7f3e3a8f7c40, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
        __arg4 = 8
        __arg2 = 139905247181888
        _a3 = 0
        _a1 = 139905247182096
        resultvar = <optimized out>
        __arg3 = 0
        __arg1 = 139905247182096
        _a4 = 8
        _a2 = 139905247181888
        resultvar = <optimized out>
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
        result = <optimized out>
        sc_ret = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        sc_cancel_oldtype = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  0x00007f3e3c3ad2bc in __sigwait (set=0x7f3e3a8f7d10, sig=0x7f3e3a8f7d00) at ../sysdeps/unix/sysv/linux/sigwait.c:28
        si = {si_signo = 0, si_errno = -1, si_code = 0, __pad0 = 0, _sifields = {_pad = {872519888, 32574, 872519816, 32574, 825110577, 858667310, 1010630708, 32574, 0, 0, 872575656, 32574, 0, 0, 1250485880, 22046, 1010672432, 32574, 1250169848, 22046, 982482144, 32574, -655010560, 579136882, 982482144, 32574, 1010639230, 32574}, _kill = {si_pid = 872519888, si_uid = 32574}, _timer = {si_tid = 872519888, si_overrun = 32574, si_sigval = {sival_int = 872519816, sival_ptr = 0x7f3e34019888}}, _rt = {si_pid = 872519888, si_uid = 32574, si_sigval = {sival_int = 872519816, sival_ptr = 0x7f3e34019888}}, _sigchld = {si_pid = 872519888, si_uid = 32574, si_status = 872519816, si_utime = 3687948015419404337, si_stime = 139905275330612}, _sigfault = {si_addr = 0x7f3e340198d0, si_addr_lsb = -26488, _bounds = {_addr_bnd = {_lower = 0x332e392e312e3031, _upper = 0x7f3e3c3d0034 <abort@plt+4>}, _pkey = 825110577}}, _sigpoll = {si_band = 139905137219792, si_fd = 872519816}, _sigsys = {_call_addr = 0x7f3e340198d0, _syscall = 872519816, _arch = 32574}}}
        ret = <optimized out>
#2  0x00007f3e3c3d2946 in triton_thread (thread=0x561e4a8413e8) at /usr/local/src/accel-ppp/accel-pppd/triton/triton.c:171
        set = {__val = {516, 0 <repeats 15 times>}}
        sig = 10
        need_free = 0
        stack = 0x0
#3  0x00007f3e3c3a2fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139905247184640, -7190569452712916160, 140723998092446, 140723998092447, 139905247184640, 0, 7082335035676003136, 7082349506948729664}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#4  0x00007f3e3bedd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 3 (Thread 0x7f3e3b17a700 (LWP 29181)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x561e48d14f68 <uc_cond+40>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
        __ret = -512
        oldtype = 0
        err = <optimized out>
        oldtype = <optimized out>
        err = <optimized out>
        __ret = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x561e48d14f00 <uc_lock>, cond=0x561e48d14f40 <uc_cond>) at pthread_cond_wait.c:502
        spin = 0
        buffer = {__routine = 0x7f3e3c3a8d80 <__condvar_cleanup_waiting>, __arg = 0x7f3e3b179c90, __canceltype = 0, __prev = 0x0}
        cbuffer = {wseq = 0, cond = 0x561e48d14f40 <uc_cond>, mutex = 0x561e48d14f00 <uc_lock>, private = 0}
        rt = <optimized out>
        err = <optimized out>
        g = 0
        flags = <optimized out>
        g1_start = <optimized out>
        signals = <optimized out>
        result = 0
        wseq = 0
        seq = 0
        private = 0
        maxspin = <optimized out>
        err = <optimized out>
        result = <optimized out>
        wseq = <optimized out>
        g = <optimized out>
        seq = <optimized out>
        flags = <optimized out>
        private = <optimized out>
        signals = <optimized out>
        g1_start = <optimized out>
        spin = <optimized out>
        buffer = <optimized out>
        cbuffer = <optimized out>
        rt = <optimized out>
        s = <optimized out>
#2  __pthread_cond_wait (cond=0x561e48d14f40 <uc_cond>, mutex=0x561e48d14f00 <uc_lock>) at pthread_cond_wait.c:655
No locals.
#3  0x0000561e48ce053a in uc_thread (unused=0x0) at /usr/local/src/accel-ppp/accel-pppd/ppp/ppp.c:350
        uc = 0x0
        fd = 0
        set = {__val = {18446744067266838271, 18446744073709551615 <repeats 15 times>}}
#4  0x00007f3e3c3a2fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139905256105728, -7190569452712916160, 140723998092542, 140723998092543, 139905256105728, 0, 7082333731079686976, 7082349506948729664}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#5  0x00007f3e3bedd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 2 (Thread 0x7f3e3b97b700 (LWP 29180)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x7f3e3bdc134c <cond+44>) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
        __ret = -512
        oldtype = 0
        err = <optimized out>
        oldtype = <optimized out>
        err = <optimized out>
        __ret = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x7f3e3bdc1360 <lock>, cond=0x7f3e3bdc1320 <cond>) at pthread_cond_wait.c:502
        spin = 0
        buffer = {__routine = 0x7f3e3c3a8d80 <__condvar_cleanup_waiting>, __arg = 0x7f3e3b976c60, __canceltype = 469868256, __prev = 0x0}
        cbuffer = {wseq = 174255, cond = 0x7f3e3bdc1320 <cond>, mutex = 0x7f3e3bdc1360 <lock>, private = 0}
        rt = <optimized out>
        err = <optimized out>
        g = 1
        flags = <optimized out>
        g1_start = <optimized out>
        signals = <optimized out>
        result = 0
        wseq = 174255
        seq = 87127
        private = 0
        maxspin = <optimized out>
        err = <optimized out>
        result = <optimized out>
        wseq = <optimized out>
        g = <optimized out>
        seq = <optimized out>
        flags = <optimized out>
        private = <optimized out>
        signals = <optimized out>
        g1_start = <optimized out>
        spin = <optimized out>
        buffer = <optimized out>
        cbuffer = <optimized out>
        rt = <optimized out>
        s = <optimized out>
#2  __pthread_cond_wait (cond=0x7f3e3bdc1320 <cond>, mutex=0x7f3e3bdc1360 <lock>) at pthread_cond_wait.c:655
No locals.
#3  0x00007f3e3bdbc930 in log_thread (unused=0x0) at /usr/local/src/accel-ppp/accel-pppd/logs/log_file.c:129
        lf = 0x561e4a84adf0
        iov = {{iov_base = 0x7f3e30049d0c, iov_len = 32}, {iov_base = 0x7f3e3001f13c, iov_len = 80}, {iov_base = 0x7f3e3003e68c, iov_len = 32}, {iov_base = 0x7f3e30049e8c, iov_len = 70}, {iov_base = 0x7f3e300494cc, iov_len = 32}, {iov_base = 0x7f3e3001e56c, iov_len = 68}, {iov_base = 0x7f3e30026c0c, iov_len = 32}, {iov_base = 0x7f3e3002968c, iov_len = 41}, {iov_base = 0x7f3e3004b9ac, iov_len = 32}, {iov_base = 0x7f3e3002fa9c, iov_len = 52}, {iov_base = 0x7f3e3000d2ec, iov_len = 32}, {iov_base = 0x7f3e30034d3c, iov_len = 25}, {iov_base = 0x7f3e1c006cdc, iov_len = 32}, {iov_base = 0x7f3e1c01c48c, iov_len = 24}, {iov_base = 0x7f3e1c0311ec, iov_len = 32}, {iov_base = 0x7f3e1c02fa1c, iov_len = 24}, {iov_base = 0x7f3e1c031a7c, iov_len = 32}, {iov_base = 0x7f3e1c00c38c, iov_len = 67}, {iov_base = 0x7f3e1c030bdc, iov_len = 32}, {iov_base = 0x7f3e1c031c8c, iov_len = 128}, {iov_base = 0x7f3e1c03096c, iov_len = 18}, {iov_base = 0x7f3e1c02be1c, iov_len = 32}, {iov_base = 0x7f3e1c03255c, iov_len = 128}, {iov_base = 0x7f3e1c03264c, iov_len = 128}, {iov_base = 0x7f3e1c02ba2c, iov_len = 128}, {iov_base = 0x7f3e1c02bb1c, iov_len = 36}, {iov_base = 0x7f3e1c02eedc, iov_len = 32}, {iov_base = 0x7f3e1c02f0ac, iov_len = 128}, {iov_base = 0x7f3e1c02ecfc, iov_len = 2}, {iov_base = 0x7f3e1c02a88c, iov_len = 32}, {iov_base = 0x7f3e1c02a58c, iov_len = 71}, {iov_base = 0x7f3e1c037cdc, iov_len = 32}, {iov_base = 0x7f3e1c02aa0c, iov_len = 40}, {iov_base = 0x7f3e1c0384ec, iov_len = 32}, {iov_base = 0x7f3e1c03815c, iov_len = 25}, {iov_base = 0x7f3e1c037dcc, iov_len = 32}, {iov_base = 0x7f3e1c02f19c, iov_len = 24}, {iov_base = 0x7f3e1c02fd5c, iov_len = 32}, {iov_base = 0x7f3e1c037f4c, iov_len = 24}, {iov_base = 0x7f3e1c02a85c, iov_len = 32}, {iov_base = 0x7f3e1c03238c, iov_len = 24}, {iov_base = 0x7f3e1c037fdc, iov_len = 32}, {iov_base = 0x7f3e1c037cdc, iov_len = 24}, {iov_base = 0x7f3e3004962c, iov_len = 47}, {iov_base = 0x7f3e3402bf9c, iov_len = 37}, {iov_base = 0x7f3e3402be1c, iov_len = 47}, {iov_base = 0x7f3e3402c4ac, iov_len = 37}, {iov_base = 0x7f3e3402c32c, iov_len = 42}, {iov_base = 0x0, iov_len = 0} <repeats 881 times>, {iov_base = 0x0, iov_len = 139905275422444}, {iov_base = 0x0, iov_len = 1042}, {iov_base = 0x7f3e3bfaa500, iov_len = 1}, {iov_base = 0x7f3e3bde6b1c, iov_len = 139905275423476}, {iov_base = 0x412, iov_len = 139905269193888}, {iov_base = 0x7f3e3bfaa500, iov_len = 139905264494728}, {iov_base = 0x7f3e3b97a884, iov_len = 139905275422814}, {iov_base = 0x0, iov_len = 139905269193888}, {iov_base = 0x7f3e3bde7fa8, iov_len = 139905275127115}, {iov_base = 0xa8428197, iov_len = 139905275118680}, {iov_base = 0x2a10a06, iov_len = 139905264494928}, {iov_base = 0x7f3e3b97a940, iov_len = 139905264494728}, {iov_base = 0x561e4a846d70, iov_len = 25}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 2822930839}, {iov_base = 0x7f3e3c3dbd80, iov_len = 139905264495080}, {iov_base = 0x7f3e3c3dba20, iov_len = 0}, {iov_base = 0x7f3e3bfab1d0, iov_len = 139905275425679}, {iov_base = 0x8, iov_len = 139905270985168}, {iov_base = 0x5, iov_len = 0}, {iov_base = 0x1, iov_len = 139905275378208}, {iov_base = 0x7f3e3c39e54b, iov_len = 4294967296}, {iov_base = 0x7f3e3b97a940, iov_len = 139905264494928}, {iov_base = 0x7f3e3c3dbd80, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0xffffffff, iov_len = 0}, {iov_base = 0x7f3e3bdee158, iov_len = 139905270981888}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 139905275228480}, {iov_base = 0x0, iov_len = 140723998092542}, {iov_base = 0x7ffcdbeaecff, iov_len = 139905264498432}, {iov_base = 0x0, iov_len = 139905275443968}, {iov_base = 0x5, iov_len = 0}, {iov_base = 0x0, iov_len = 139905269162328}, {iov_base = 0x7f3e3b97ada0, iov_len = 139905275471034}, {iov_base = 0x7f3e3b97ae48, iov_len = 139905275146025}, {iov_base = 0x7f3e3b97add0, iov_len = 24}, {iov_base = 0x7f3e3b97add0, iov_len = 139905264498432}, {iov_base = 0x7f3e3b97b700, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 281470681751424}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0x7f3e3bfa0560 <_nl_global_locale>, iov_len = 139905270954240}, {iov_base = 0x1000, iov_len = 0}, {iov_base = 0x800000, iov_len = 0}, {iov_base = 0xff0000, iov_len = 4278190080}, {iov_base = 0xa090a090a090a09, iov_len = 723120249109023241}, {iov_base = 0xa090a090a090a09, iov_len = 723120249109023241}, {iov_base = 0x0, iov_len = 0}, {iov_base = 0xa090a090a090a09, iov_len = 723120249109023241}, {iov_base = 0xffffffff, iov_len = 1099511627520}, {iov_base = 0x0, iov_len = 0} <repeats 12 times>, {iov_base = 0x2, iov_len = 0}, {iov_base = 0x0, iov_len = 0} <repeats 21 times>}
        chunk = 0x7f3e1c01f7a0
        msg = 0x7f3e1c019f18
        iov_cnt = 16
        msg_list = {next = 0x7f3e3b976d00, prev = 0x7f3e3b976d00}
        free_list = {next = 0x7f3e3b976d10, prev = 0x7f3e3b976d10}
        set = {__val = {18446744067266838271, 18446744073709551615 <repeats 15 times>}}
#4  0x00007f3e3c3a2fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
        ret = <optimized out>
        pd = <optimized out>
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139905264498432, -7190569452712916160, 140723998092542, 140723998092543, 139905264498432, 0, 7082332632104930112, 7082349506948729664}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#5  0x00007f3e3bedd4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
No locals.

Thread 1 (Thread 0x7f3e3bde2cc0 (LWP 29179)):
#0  0x00007f3e3be1c4cc in __GI___sigtimedwait (set=set@entry=0x7ffcdbeaee10, info=info@entry=0x7ffcdbeaed20, timeout=timeout@entry=0x0) at ../sysdeps/unix/sysv/linux/sigtimedwait.c:29
        __arg4 = 8
        __arg2 = 140723998092576
        _a3 = 0
        _a1 = 140723998092816
        resultvar = <optimized out>
        __arg3 = 0
        __arg1 = 140723998092816
        _a4 = 8
        _a2 = 140723998092576
        resultvar = <optimized out>
        sc_cancel_oldtype = 0
        sc_ret = <optimized out>
        result = <optimized out>
        sc_ret = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        sc_cancel_oldtype = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg4 = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a4 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
#1  0x00007f3e3c3ad2bc in __sigwait (set=0x7ffcdbeaee10, sig=0x7ffcdbeaede4) at ../sysdeps/unix/sysv/linux/sigwait.c:28
        si = {si_signo = 0, si_errno = 0, si_code = 0, __pad0 = 0, _sifields = {_pad = {0 <repeats 20 times>, 1250194960, 22046, -655010560, 4, 1249982856, 22046, 1250485864, 22046}, _kill = {si_pid = 0, si_uid = 0}, _timer = {si_tid = 0, si_overrun = 0, si_sigval = {sival_int = 0, sival_ptr = 0x0}}, _rt = {si_pid = 0, si_uid = 0, si_sigval = {sival_int = 0, sival_ptr = 0x0}}, _sigchld = {si_pid = 0, si_uid = 0, si_status = 0, si_utime = 0, si_stime = 0}, _sigfault = {si_addr = 0x0, si_addr_lsb = 0, _bounds = {_addr_bnd = {_lower = 0x0, _upper = 0x0}, _pkey = 0}}, _sigpoll = {si_band = 0, si_fd = 0}, _sigsys = {_call_addr = 0x0, _syscall = 0, _arch = 0}}}
        ret = <optimized out>
#2  0x0000561e48d05eeb in main (_argc=5, _argv=0x7ffcdbeaf018) at /usr/local/src/accel-ppp/accel-pppd/main.c:389
        set = {__val = {16386, 0 <repeats 15 times>}}
        i = 5
        sig = 22046
        goto_daemon = 0
        len = 32764
        pid = 0
        sa = {__sigaction_handler = {sa_handler = 0x561e48d052d9 <sigsegv>, sa_sigaction = 0x561e48d052d9 <sigsegv>}, sa_mask = {__val = {18446744067267100671, 18446744073709551615 <repeats 15 times>}}, sa_flags = 0, sa_restorer = 0x0}
        pagesize = 4096
        internal = 0
        no_sigint = 0
        no_sigsegv = 0
warning: target file /proc/29179/cmdline contained unexpected null characters
Saved corefile /media/dlogs/coredump-script-2020-07-31-23-36-20-839756840
Detaching from program: /usr/sbin/accel-pppd, process 29179
[Inferior 1 (process 29179) detached]
2020-08-01-06-04-47-551749750 accelppp crashed/stopped now
error according to systemd

Code: Select all

Aug 01 06:04:47 bng accelppp-exo[29179]: md.c: Debug-eXO: (). Function md_thread(). n < 0. errno = Bad file descriptor
Aug 01 06:04:49 bng systemd[1]: accel-ppp.service: Main process exited, code=exited, status=255/EXCEPTION
Aug 01 06:04:49 bng systemd[1]: accel-ppp.service: Failed with result 'exit-code'.
Aug 01 06:04:49 bng systemd[1]: accel-ppp.service: Service RestartSec=100ms expired, scheduling restart.
Aug 01 06:04:49 bng systemd[1]: accel-ppp.service: Scheduled restart job, restart counter is at 4.
last lines of debug.log before the crash

Code: Select all

[2020-08-01 06:04:39.275] l2tp tunnel 584-30262 (masked_ip1:62949): retransmit (timeout) [L2TP tid=30262 sid=0 <Message-Type Hello>]
[2020-08-01 06:04:39.786] l2tp tunnel 55127-43772 (masked_ip2:37947): retransmission #1
[2020-08-01 06:04:39.786] l2tp tunnel 55127-43772 (masked_ip2:37947): retransmit (timeout) [L2TP tid=43772 sid=0 <Message-Type Hello>]
[2020-08-01 06:04:40.274] : ddbe02d5f0f74d17: recv [RADIUS(1) Access-Reject id=1 <MS-CHAP-Error "^AE=691 R=1 C=masked_hash V=3 M=Authentication rejected">]
[2020-08-01 06:04:40.274] : ddbe02d5f0f74d17: send [MSCHAP-v2 Failure id=1 "E=691 R=0 V=3 M=Authentication failure"]
[2020-08-01 06:04:40.274] : ddbe02d5f0f74d17: masked_user@exo.cat: authentication failed
[2020-08-01 06:04:40.274] masked_user@exo.cat: authentication failed
[2020-08-01 06:04:40.274] : ddbe02d5f0f74d17: send [LCP TermReq id=62]
[2020-08-01 06:04:40.281] : ddbe02d5f0f74d17: recv [LCP TermReq id=3]
[2020-08-01 06:04:40.281] : ddbe02d5f0f74d17: send [LCP TermAck id=3]
[2020-08-01 06:04:40.285] l2tp tunnel 28856-8314 (masked_ip3:1701): recv [L2TP tid=28856 sid=15428 <Message-Type Call-Disconnect-Notify> <Result-Code> <Assigned-Session-ID 7578>]
[2020-08-01 06:04:40.285] l2tp session 28856-8314, 15428-7578: handling CDN
[2020-08-01 06:04:40.285] l2tp session 28856-8314, 15428-7578: CDN received from peer (result: 1, error: 0), disconnecting session
[2020-08-01 06:04:40.285] l2tp session 28856-8314, 15428-7578: deleting session
[2020-08-01 06:04:40.285] l2tp session 28856-8314, 15428-7578: deleting data channel
[2020-08-01 06:04:40.285] l2tp tunnel 28856-8314 (masked_ip3:1701): no more session, disconnecting tunnel
[2020-08-01 06:04:40.285] l2tp tunnel 28856-8314 (masked_ip3:1701): sending StopCCN (res: 1, err: 0)
[2020-08-01 06:04:40.285] l2tp tunnel 28856-8314 (masked_ip3:1701): send [L2TP tid=8314 sid=0 <Message-Type Stop-Ctrl-Conn-Notify> <Assigned-Tunnel-ID 28856> <Result-Code>]
[2020-08-01 06:04:40.286] : ddbe02d5f0f74d17: session destroyed
[2020-08-01 06:04:40.286] l2tp session 28856-8314, 15428-7578: session destroyed
[2020-08-01 06:04:40.291] l2tp tunnel 28856-8314 (masked_ip3:1701): tunnel disconnection acknowledged by peer, deleting tunnel
[2020-08-01 06:04:40.291] l2tp tunnel 28856-8314 (masked_ip3:1701): deleting tunnel
[2020-08-01 06:04:40.291] l2tp tunnel 28856-8314 (masked_ip3:1701): tunnel destroyed
[2020-08-01 06:04:43.966] l2tp tunnel 51758-60801 (masked_ip4:29608): retransmission #1
[2020-08-01 06:04:43.966] l2tp tunnel 51758-60801 (masked_ip4:29608): retransmit (timeout) [L2TP tid=60801 sid=0 <Message-Type Hello>]
[2020-08-01 06:04:46.421] l2tp: recv [L2TP tid=0 sid=0 <Message-Type Start-Ctrl-Conn-Request> <Protocol-Version 256> <Framing-Capabilities 3> <Bearer-Capabilities 0> <Firmware-Revision 1680> <Host-Name OpenWrt> <Vendor-Name xelerance.com> <Assigned-Tunnel-ID -26004> <Recv-Window-Size 4>]
[2020-08-01 06:04:46.422] l2tp: handling SCCRQ from masked_ip3
[2020-08-01 06:04:46.422] l2tp: new tunnel 8104-39532 created following reception of SCCRQ from masked_ip3:1701
[2020-08-01 06:04:46.422] l2tp tunnel 8104-39532 (masked_ip3:1701): sending SCCRP
[2020-08-01 06:04:46.422] l2tp tunnel 8104-39532 (masked_ip3:1701): send [L2TP tid=39532 sid=0 <Message-Type Start-Ctrl-Conn-Reply> <Protocol-Version 256> <Host-Name bng.exo.cat> <Framing-Capabilities 3> <Assigned-Tunnel-ID 8104> <Vendor-Name accel-ppp> <Recv-Window-Size 16>]
[2020-08-01 06:04:46.429] l2tp tunnel 8104-39532 (masked_ip3:1701): recv [L2TP tid=8104 sid=0 <Message-Type Start-Ctrl-Conn-Connected>]
[2020-08-01 06:04:46.429] l2tp tunnel 8104-39532 (masked_ip3:1701): handling SCCCN
[2020-08-01 06:04:46.429] l2tp tunnel 8104-39532 (masked_ip3:1701): established at masked_l2tp_server:1701
[2020-08-01 06:04:46.439] l2tp tunnel 8104-39532 (masked_ip3:1701): recv [L2TP tid=8104 sid=0 <Message-Type Incoming-Call-Request> <Assigned-Session-ID 858> <Call-Serial-Number 22691> <Bearer-Type 0>]
[2020-08-01 06:04:46.439] l2tp tunnel 8104-39532 (masked_ip3:1701): handling ICRQ
[2020-08-01 06:04:46.439] l2tp session 8104-39532, 51086-858: sending ICRP
[2020-08-01 06:04:46.440] l2tp tunnel 8104-39532 (masked_ip3:1701): new session 51086-858 created following reception of ICRQ
[2020-08-01 06:04:46.440] l2tp tunnel 8104-39532 (masked_ip3:1701): send [L2TP tid=39532 sid=858 <Message-Type Incoming-Call-Reply> <Assigned-Session-ID -14450>]
[2020-08-01 06:04:46.447] l2tp tunnel 8104-39532 (masked_ip3:1701): recv [L2TP tid=8104 sid=51086 <Message-Type Incoming-Call-Connected> <TX-Speed 0> <Framing-Type 1> <RX-Speed 0>]
[2020-08-01 06:04:46.447] l2tp session 8104-39532, 51086-858: handling ICCN
[2020-08-01 06:04:46.447] : : starting data channel for l2tp(masked_ip3:1701 session 8104-39532, 51086-858)
[2020-08-01 06:04:46.448] : ddbe02d5f0f74d18: send [LCP ConfReq id=be <auth MSCHAP-v2> <mru 1450> <magic masked_magic1>]
[2020-08-01 06:04:46.491] : ddbe02d5f0f74d18: recv [LCP ConfReq id=1 <mru 1420> < 2 6 0 0 0 0 > <magic masked_magic2>]
[2020-08-01 06:04:46.491] : ddbe02d5f0f74d18: send [LCP ConfRej id=1  < 2 6 0 0 0 0 >]
[2020-08-01 06:04:46.499] : ddbe02d5f0f74d18: recv [LCP ConfReq id=2 <mru 1420> <magic masked_magic2>]
[2020-08-01 06:04:46.499] : ddbe02d5f0f74d18: send [LCP ConfAck id=2 ]
[2020-08-01 06:04:46.551] l2tp tunnel 8104-39532 (masked_ip3:1701): recv [L2TP tid=8104 sid=0 <Message-Type Stop-Ctrl-Conn-Notify> <Assigned-Tunnel-ID -26004> <Result-Code>]
[2020-08-01 06:04:46.551] l2tp tunnel 8104-39532 (masked_ip3:1701): handling StopCCN
[2020-08-01 06:04:46.551] l2tp tunnel 8104-39532 (masked_ip3:1701): StopCCN received from peer (result: 1, error: 0, message: "Goodbye!"), disconnecting tunnel
[2020-08-01 06:04:46.551] : ddbe02d5f0f74d18: recv [LCP TermReq id=3]
[2020-08-01 06:04:46.551] : ddbe02d5f0f74d18: send [LCP TermAck id=3]
[2020-08-01 06:04:46.551] l2tp session 8104-39532, 51086-858: deleting session
[2020-08-01 06:04:46.551] : ddbe02d5f0f74d18: session destroyed
[2020-08-01 06:04:49.561] accel-ppp version f09b73c6c96688eb23bc4c408ffb9dc10fb39dcd

[1] https://gitlab.com/guifi-exo/accel-ppp/ ... ervice#L13
[2] https://gitlab.com/guifi-exo/accel-ppp/ ... ppp.sh#L38
[3] https://github.com/xebd/accel-ppp/blob/ ... ile.c#L269
dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: accelppp exits unexpectedly with log error md:epoll_wait

Post by dimka88 »

Hello exopedro,
Can you try the following patch

Code: Select all

diff --git a/accel-pppd/logs/log_file.c b/accel-pppd/logs/log_file.c
index 6eafd42..e5ccc13 100644
--- a/accel-pppd/logs/log_file.c
+++ b/accel-pppd/logs/log_file.c
@@ -265,8 +265,10 @@ static void general_log(struct log_target_t *t, struct log_msg_t *msg, struct ap
 static struct ap_private *find_pd(struct ap_session *ses, void *pd_key)
 {
 	struct ap_private *pd;
+	struct list_head *pos, *next;
 
-	list_for_each_entry(pd, &ses->pd_list, entry) {
+	list_for_each_safe(pos, next, &ses->pd_list) {
+		pd = list_entry(pos->next, typeof(*pd), entry);
 		if (pd->key == pd_key) {
 			return pd;
 		}
@@ -551,8 +553,8 @@ static void ev_ctrl_started(struct ap_session *ses)
 		}
 		memset(fpd, 0, sizeof(*fpd));
 		fpd->pd.key = &pd_key3;
-		list_add_tail(&fpd->pd.entry, &ses->pd_list);
 		INIT_LIST_HEAD(&fpd->msgs);
+		list_add_tail(&fpd->pd.entry, &ses->pd_list);
 	}
 }
 
exopedro
Posts: 8
Joined: 23 Jun 2020, 15:17

Re: accelppp exits unexpectedly with log error md:epoll_wait

Post by exopedro »

Hi dimka88,

we tested your patch in a test environment: apparently works for one l2tp tunnel. But per-user-dir logs are not working. So we are stopping our testing here. We prefer to wait until your next feedback. When per-user-dir works, our plan is to deploy it in production to see if we still suffer from this bug.

now we are working in a new branch called exo, which is 1.12.0, plus your patch and some internal tooling we found interesting for us [1]

and here is your patch applied to the exo branch [2]

[1] https://gitlab.com/guifi-exo/accel-ppp/-/commits/exo

[2] https://gitlab.com/guifi-exo/accel-ppp/ ... 37b7b16d2e
exopedro
Posts: 8
Joined: 23 Jun 2020, 15:17

Re: accelppp exits unexpectedly with log error md:epoll_wait

Post by exopedro »

for people hitting this bug, this is becoming like a "known bug" meanwhile, right? We are succesfully running accelppp nicely since 3 weeks

Code: Select all

   Active: active (running) since Sun 2020-08-02 17:20:05 CEST; 3 weeks 3 days ago
what we are doing is blocking with a firewall the former clients that are insistently trying to connect to the L2TP server (in openwrt clients could happen each 15 seconds). Hence, that prooves that these former clients logging causes accel-ppp to crash

(of course you are not going to hit this bug if you are using an authentication link layer such as GPON)
exopedro
Posts: 8
Joined: 23 Jun 2020, 15:17

Re: accelppp exits unexpectedly with log error md:epoll_wait

Post by exopedro »

Hi dimka88,

Victor from eXO found a way to fix your patch [1], so now it seems it works with per-user logging. He changed the first argument of the function list_entry that was pos->next and now is pos. Can you verify-test that this is correct?

Thanks,
Victor and Pedro

[1]

Code: Select all

diff --git a/accel-pppd/logs/log_file.c b/accel-pppd/logs/log_file.c
--- a/accel-pppd/logs/log_file.c
+++ b/accel-pppd/logs/log_file.c
@@ -265,8 +265,10 @@ static void general_log(struct log_target_t *t, struct log_msg_t *msg, struct ap
 static struct ap_private *find_pd(struct ap_session *ses, void *pd_key)
 {
 	struct ap_private *pd;
+	struct list_head *pos, *next;
 
-	list_for_each_entry(pd, &ses->pd_list, entry) {
+	list_for_each_safe(pos, next, &ses->pd_list) {
+		pd = list_entry(pos, typeof(*pd), entry);
 		if (pd->key == pd_key) {
 			return pd;
 		}
@@ -551,8 +553,8 @@ static void ev_ctrl_started(struct ap_session *ses)
 		}
 		memset(fpd, 0, sizeof(*fpd));
 		fpd->pd.key = &pd_key3;
-		list_add_tail(&fpd->pd.entry, &ses->pd_list);
 		INIT_LIST_HEAD(&fpd->msgs);
+		list_add_tail(&fpd->pd.entry, &ses->pd_list);
 	}
 }

 
Post Reply