erratic accounting updates after 47 days of uptime

PPPoE related questions
hashbang
Posts: 135
Joined: 12 Jul 2015, 10:28

erratic accounting updates after 47 days of uptime

Post by hashbang »

HI,
Running accel-ppp on centos 7 with 3500 users and 5.5gig bw. Some strange behaviour in accounting updates. It stops sending accounting updates and it doesnt stops every user.
As our AAA removes users who are not sending interim updates considering as stale sessions. So I have in accurate online users in AAA . It keeps on fluctuating. After some work It was found that accounting updates are not being sent. This was checked it with tcpdump. So before rebooting I thought i should post it in forum for some help.

thanks
dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: erratic accounting updates after 47 days of uptime

Post by dimka88 »

Hi hashbang
In accel-ppp.log you can check is acct requests were sent.
Did you see some strange in dmesg about DROP or table full?
You can join to the English speaking community https://t.me/accel_ppp_en
hashbang
Posts: 135
Joined: 12 Jul 2015, 10:28

Re: erratic accounting updates after 47 days of uptime

Post by hashbang »

hi
I got a call for the same problem. I logged into server and saw logs stoppped at 10:19 no more logs and after few minutes the logs started flooding. I have other accel-ppp server along with this that one is running fine. Logs are :
[2020-04-26 10:19:19]: info: snpjatin1: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.1702: snpsanjiv: second session denied
[2020-04-26 10:19:19]: info: ens2f0.1702: snpsanjiv: authentication failed
[2020-04-26 10:19:19]: info: snpsanjiv: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.1702: snpgaurav7: second session denied
[2020-04-26 10:19:19]: info: ens2f0.1702: snpgaurav7: authentication failed
[2020-04-26 10:19:19]: info: snpgaurav7: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.1702: snpchander: second session denied
[2020-04-26 10:19:19]: info: ens2f0.1702: snpchander: authentication failed
[2020-04-26 10:19:19]: info: snpchander: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.1702: snpverma3: second session denied
[2020-04-26 10:19:19]: info: ens2f0.1702: snpverma3: authentication failed
[2020-04-26 10:19:19]: info: snpverma3: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.1702: Snpjprashant: second session denied
[2020-04-26 10:19:19]: info: ens2f0.1702: Snpjprashant: authentication failed
[2020-04-26 10:19:19]: info: Snpjprashant: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.1702: Snpchahal2: second session denied
[2020-04-26 10:19:19]: info: ens2f0.1702: Snpchahal2: authentication failed
[2020-04-26 10:19:19]: info: Snpchahal2: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.1703: snpmohit6: second session denied
[2020-04-26 10:19:19]: info: ens2f0.1703: snpmohit6: authentication failed
[2020-04-26 10:19:19]: info: snpmohit6: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.113: nhsunil: second session denied
[2020-04-26 10:19:19]: info: ens2f0.113: nhsunil: authentication failed
[2020-04-26 10:19:19]: info: nhsunil: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.113: nhamit: second session denied
[2020-04-26 10:19:19]: info: ens2f0.113: nhamit: authentication failed
[2020-04-26 10:19:19]: info: nhamit: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.515: nrlpony: second session denied
[2020-04-26 10:19:19]: info: ens2f0.515: nrlpony: authentication failed
[2020-04-26 10:19:19]: info: nrlpony: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.103: tinyvishnu33: second session denied
[2020-04-26 10:19:19]: info: ens2f0.103: tinyvishnu33: authentication failed
[2020-04-26 10:19:19]: info: tinyvishnu33: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.132: kgravinder: second session denied
[2020-04-26 10:19:19]: info: ens2f0.132: kgravinder: authentication failed
[2020-04-26 10:19:19]: info: kgravinder: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.516: bvbeersingh: second session denied
[2020-04-26 10:19:19]: info: ens2f0.516: bvbeersingh: authentication failed
[2020-04-26 10:19:19]: info: bvbeersingh: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.516: bvmanoj8: second session denied
[2020-04-26 10:19:19]: info: ens2f0.516: bvmanoj8: authentication failed
[2020-04-26 10:19:19]: info: bvmanoj8: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.1513: mparmender: second session denied
[2020-04-26 10:19:19]: info: ens2f0.1513: mparmender: authentication failed
[2020-04-26 10:19:19]: info: mparmender: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.1511: snadity: second session denied
[2020-04-26 10:19:19]: info: ens2f0.1511: snadity: authentication failed
[2020-04-26 10:19:19]: info: snadity: authentication failed
[2020-04-26 10:19:19]: warn: pppoe810: radius: server(1) not responding, terminating session...
[2020-04-26 10:19:19]: info: ens2f0.132: hkdavender: second session denied
[2020-04-26 10:19:19]: info: ens2f0.132: hkdavender: authentication failed
[2020-04-26 10:19:19]: info: hkdavender: authentication failed
[2020-04-26 10:19:19]: info: ens2f0.622: rkjalal: second session denied
[2020-04-26 10:19:19]: info: ens2f0.622: rkjalal: authentication failed
[2020-04-26 10:19:19]: info: rkjalal: authentication failed

thanks
dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: erratic accounting updates after 47 days of uptime

Post by dimka88 »

Hi, you need to check all logs like journalctl and dmesg.
What in [radius] section?
hashbang
Posts: 135
Joined: 12 Jul 2015, 10:28

Re: erratic accounting updates after 47 days of uptime

Post by hashbang »

Hi,
nothing in journalctl and dmesg. I by mistake replied into other post. Definitely there is problem with accounting updates. I have two servers running accel-ppp and both are giving same problem. Suddenly accounting updates becomes erratic. Just now i got a complaint. Couldnt find any thing in dmesg journalctl. I have other 10 nas running (MIkrotik) So its not the radius problem. I just changed the interim update time in radius setting from 300 to 600 then accel-cmd reload and accouting updates started going perfectly. I'm checking with the tcpump (amount of packets suddenly increases).

thanks
dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: erratic accounting updates after 47 days of uptime

Post by dimka88 »

Hi, set all req-limit to 0 in radius section to 0, it seems this is an issue.
Do you have an idea of how to reproduce this in LAB?
hashbang
Posts: 135
Joined: 12 Jul 2015, 10:28

Re: erratic accounting updates after 47 days of uptime

Post by hashbang »

Hi,
TY, As its a live server so not possible to reproduce in lab. I'll change the config as per ur suggestion and update. In the meanwhile i'd started generating graphs of interim updates
hashbang
Posts: 135
Joined: 12 Jul 2015, 10:28

Re: erratic accounting updates after 47 days of uptime

Post by hashbang »

HI,
Did both the servers by changing req-limit=0. 1st server went very well after reload but 2nd server started giving same problem. Stats show no interim-update sent. Plz see the screen shot. One thing more accel-cmd pppoe interface show different online users and accel-cmd show sessions | grep active | wc -l showed other value 32xxx and 17xxx respectively
Screenshot from 2020-05-12 01-24-36.png
Screenshot from 2020-05-12 01-24-36.png (160.64 KiB) Viewed 3983 times
thanks
hashbang
Posts: 135
Joined: 12 Jul 2015, 10:28

Re: erratic accounting updates after 47 days of uptime

Post by hashbang »

Its all messed up no interim updates going
did reload
did terminat all
restart accel-ppp
accel-cmd show stat
interim-update counter stuck at 18 and its bee 30 minutes after restart
again did restart the accel-ppp service
now till now updates are going fine

radius(1, 10):
state: active
fail count: 0
request count: 0
queue length: 0
auth sent: 4574
auth lost(total/5m/1m): 48/0/0
auth avg query time(5m/1m): 824/949 ms
acct sent: 7597
acct lost(total/5m/1m): 3229/16/0
acct avg query time(5m/1m): 229/12 ms
interim sent: 2701
interim lost(total/5m/1m): 7/7/0
interim avg query time(5m/1m): 1/0 ms
dimka88
Posts: 866
Joined: 13 Oct 2014, 05:51
Contact:

Re: erratic accounting updates after 47 days of uptime

Post by dimka88 »

hashbang I need logs, [log]level=5 [radius]interim-verbose=1
Please, add also [radius]acct-interim-jitter=60 read more https://accel-ppp.readthedocs.io/en/lat ... adius.html
Post Reply