[{FH}Pear]

Servers are currently down for a short maintenance period.

I'm working with our host to address the massive lag spikes that we have been experiencing. The issue appears to be related to disk IO, and I forwarded my findings to our host. At this time, our server instance is being migrated to a different host machine to see if the different hardware helps solve the lag spikes. I will post here again when the migration is complete.

[{FH}Pear]

The migration is now complete, and the servers are back up.

If you feel that there are still lag spikes not related to network performance, please report them here. I'll be watching for any lingering issues as well, and will further progress troubleshooting with our provider if needed.

[kilrbkild]

Nice job Mate! Looks like you got it! It's been running great, lag free, for the past few hours! Very smooth   

[Apoc]

Does the server seem sticky to any1 else ...?

[{FH}Pear]

Does the server seem sticky to any1 else ...?

Can you be a little bit more descriptive? I'm not sure I understand what your trying to describe, do you just feel like your player model moves slower?

[Cryptocosis]

I was playing the other day and getting lag spikes, I'd shoot a player and my screen would freeze my lag meter went nuts and boom picked right back up. I'll make a demo and show you pear. Kick ass work btw

[Apoc]

sry pear i meant it feels not so much laggy but like your trying to run in jello when the server is under load. some ppl have been lagging but i lag all the time anyway ping wise. how much Ram is being used while under load? could it be a ram thing.

[{FH}spankeem]

I demoed some people doing moon walks during lag issues yesterday.  Ill post when I get home

[Apoc]

Does the server seem sticky to any1 else ...?

do you just feel like your player model moves slower?

yeah mate a little .esp when turning left and right .. it could just be on my side pear idk

[{FH}Pear]

Could also be perception, played for a few weeks on a stuttering server and now its not quite as bad...

Processor usage is fine, more than enough RAM etc. That was all part of my original check when I started troubleshooting. :P
You could always see how much of a role packet transit is playing (always makes me sad when I look at mine): http://fhclan.com/forum/index.php?action=downloads;sa=view;down=14


Outside of that, the issue that I was seeing on the server side seems to have dissipated for the most part. For those who are wondering/care, disk latency was spiking when trying to read and write to C:\$BitMap, the spikes were averaging about 920ms. Being a system file that keeps track of which used/free clusters, everything else essentially queued behind it until that file read/write was completed, basically choking the entire system.

I personally haven't experienced anymore issues since we migrated to new hardware.... However I'm not on 24/7 so that doesn't mean its not happening. The tricky bit is distinguishing network causes from the server side causes. If you run a MTR when you experience this more, the results will help me distinguish between the two. Times will also be helpful, the least I can do is check system logs for interrupts and such.

The last thing I can do is wipe the disk and put a fresh OS on there, I'd probably go with Linux this time around. Unfortunately such a task (regardless of OS choice) would require roughly 30 minutes to an hour of downtime, not something I would look forward to doing unless necessary.

[{FH}spankeem]

Hey Pear - is MTR a command in console - Im more than happy to provide you with those results.

[{FH}Pear]

Hey Spank, I uploaded the tool and linked it here!

You could always see how much of a role packet transit is playing (always makes me sad when I look at mine): http://fhclan.com/forum/index.php?action=downloads;sa=view;down=14