Tech - Significant performance degradation going above 14 boxes (1 Viewer)

[mattacm]

So I've been 12 boxing for a while and decided to amp it up to 18 boxing and it becomes absolutely unplayable. Here's what I've discovered so far as well as baseline information:
4770k
Dual 970 GPUs
32gb memory
isboxer/wineq
Multiple samsung pro SSDs

Performance:
30+ FPS per EQ process when playing 1-14 accounts
20-24 FPS if running 15-16 with significant stuttering
4-8 FPS when running 16+ and the entire system is on its knees. Every click takes 5-10 seconds to register.

The kicker is that I cannot locate any bottle necks in system resources. When all processes are running my CPUs average 35% and the load is evenly spread amongst the available cores. The memory is at 40% allocated. The GPU core, memory, memory controller and video engine are all 10-40% for each GPU. All temps are low and healthy. I'm also able to churn 15-20k IOPS with an IO load test while the system is otherwise crawling. I'm truly at a loss for why I'm running into a performance cliff and am hoping there are other multi boxers who have leaped this hurdle. Some ideas about potential issue:
File/folder contention within the EQ directory and associated files? - Is there a way to run accounts from different directories? If so I can load balance 6 accounts per directory structure.
CPU wait time? - While utilization and overall load is acceptable at a per core and average level there still may be a critical breaking point of CPU wait time but I'm doubtful as that will generally inflate the CPU utilization metrics

I've been tweaking graphics/cpu/performance settings and trying different things for two days now with no improvements anywhere. Any help would be greatly appreciated and if you're the person that finds my silver bullet I'll gladly toss a krono your way

 

[Razkle]

step 1 join Redguides as a level 2 member there are multiple write-ups on system tuning
step 2 make sure you only have 1 thing throttling gpu/cpu/fps

I frequently run 18-24 myself on a lesser machine than yours

 

[Redbot]

I'm not a big multiboxer, I only run 3, but I'll happily guess. I'd suggest MQ2FPS or MQ2ViewPort, but not sure if you can access those. However, you can manually set your viewport with the /viewport command. But you say your GPU isn't the bottleneck, so I doubt that's the fix.


drivecrash just posted a good guide for manually controlling your multi-core processor:
http://www.redguides.com/community/...lti-Core-processor-easily?p=254932#post254932
But again, you say it's spread evenly throughout your cores.

I find this guide good for tweaking:
http://forums.eqfreelance.net/index.php?topic=2958.0
But you say you've done most tweaks!

Here's one thing I'd try: Make a copy of your EQ folder and put it on your 2nd drive, and try splitting your sessions up. Your IOPS might be healthy because it's reading from your other drives, while EQ is killing just one.

 

[scotsbiker]

not sure why your having such problems, with that system spec you should be able to run 20-25+ easily.

i run a
i5-4670k
32gb memory
2gb 650i video card
2tb hdrive
Win 10 str8 out the box.

nothing has been tweaked and easily run 20-22.

however i have found out a few things when comparing it to my lower spec box sytems.

a. GOOGLE can drasticly slow system down. so much so ive removed it from my lower spec machines.
b. virus program slows it down considerably unless you set it to ignore EQ and Mq2 folders.
c. running instances from different drives can slow them down. i.e have eq and mq2 on same drive.

hope some of that helps

would also try running them without isboxer and wineq and see if that makes any difference

 

[Akatora]

try this

go into your eqclient.ini and do

CPUAffinity1=-1

and go through all the numbers for how many boxes you have and do -1 that will let it utilize all the cores you have instead of a single one

Thats probably whats bogging down your system, honestly I had a similar issue and this fixed it, not utilizing all cores for each instance would bog it down so bad i would get a bluescreen of death lol

i run 18 as well... and after doing that I could run them for days with no problem

 

[mattacm]

A few updates:
I have tried both explicitly setting affinity as well as -1 affinity on each instance, no noticeable difference between settings
I have shut down all conflicting applications I could find (anti malware, anti virus, backups etc)
I have ran io loads to the same drive EQ is on and get great metrics during the crawl but will try isolating EQ onto a dedicated ssd later today

Sounds like there may be more info in the level 2 section. I'll have to see what's there.
Can you expound on what throttling configurations you're referring to?

 

[Randyleo]

I gifted you a month of level 2 access. I hope it helps you out.

 

[mattacm]

Thank you, that's very kind of you. I appreciate it.

 

[Randyleo]

No problem, I just wanted to help you get it straightened out.

 

[playj]

Max Background FPS is set to Min.CPU right? Options -> Display -> Advanced

I too sometimes run into issues going past 12-13 EQ instances on a machine that can usually handle 18-20. Usually it happens after a patch when the WinEQ's EQPlayNice is not ready yet.

However, your case is different since your CPU usage is not going past 50%, so my theory is that the culprit is somewhere in your network connection (card, connection itself maybe). While each EQ instance does not take too much bandwith (16 instances only taking 50-100Kbps), there might be some QoS issues at your router (or worse at ISP level, it may not like too many open UDP ports).

THis is just a theory, so maybe start by checking on what's happening in Network Activity in Resource Monitor.

EDIT: Also turn off Windows Search Indexing (doesn't seem to be the culprit but it is one more thing to cross off).

 

[ojlojl]

sed -i '/\[Defaults\]/a Sound\=0' eqclient*.ini

 

[Akatora]

Have you tried loading up all the accounts without MQ2? If so how did that go?

and loading them up at the same time?

 

[DriveCrash]

sed -i '/\[Defaults\]/a Sound\=0' eqclient*.ini

Pretty much this. Even if you have sound turned off, each EQ Client is still activating sound card DLL files. Unless you disable sound completely for your addon boxes, each client is polling the sound card IRQ. Even though IRQ's are dynamic now, EQ doesn't understand that (it's old) so it polls directly to IRQ.

The stutter effect you see is the IRQ wait hangs as the requests for sound information get piled up. Once you get to a certain point, the IRQ requests never catch up, and the performance goes to it's knees. You are not seeing this effect your IOPS because it's not the CPU being assaulted, it's the northbridge to the soundcard bus pipeline, which IOPS testing doesn't hit.

[DEFAULTS]
Sound=0

Add or change the sound= in EQClient.ini and you should find a massive increase in performance.

Good luck & Happy boxing!

 

[mattacm]

I'll reply back with some proper responses when I can tinker more.

Drivecrash, I have added the nosound.txt file to the root directory. Does that apply similar to the above Sound=0 config in the .ini file?
Akatora, I have tried launching with every combination of ISBoxer/MQ/WinEQ with different throttling behaviors. The best I've been able to get to is about 14 FPS in an isolated dungeon with no other character models.
Playj, my boxing applications are up to date. I can run a 50+mb speed test during "the crawl" and am hard wired on a gigabit network. Because non EQ or internet based computer performance is affected (Navigating file directories, start menu, other local behavior etc.) I would be hesitant to think routing/FW is at play.

You guys have been fantastic and very willing to help. I sincerely appreciate it.

 

[Hihi]

You only have one FPS limiter in play correct? Using a combination of MQ2FPS, eqplaynice setting in WinEQ, in game, and possibly in ISBoxer can cause issues.

 

[mattacm]

Right I have tried running it with each one independently and mixing some. The sweet spot so far has been fps limiting in isboxer, rendering limiting in eqplynice (wineq). Is there a thread that addresses " optimal" fps limiting settings with the varied programs? With applications like isboxer that don't have an explicit enable/disable toggle (at least that I could find) what is considered "disabled"? A higher fps number than what the other applications are trying? 0?

- - - Updated - - -

Here's some items from my client.ini that may be at play.
GraphicsMemoryModeSwitch=1
APVOptimizations=TRUE
DisableWestBugFix=0
Sound=0
TextureQuality=3
VertexShaders=1
20PixelShaders=TRUE
14PixelShaders=TRUE
1xPixelShaders=TRUE
MultiPassLighting=FALSE
PostEffects=FALSE
UseLitBatches=TRUE
InvWinLabels=TRUE
ShowDynamicLights=FALSE
VideoMode=0
ShowSpellEffects=1
CombatMusic=1
ShowGrass=0
NetStat=FALSE
Sound44k=0
SpeakerType=0
LoadSocialAnimations=FALSE
Music=0
SoundVolume=0
SpellParticleOpacity=1.000000
EnvironmentParticleOpacity=1.000000
ActorParticleOpacity=1.000000
SpellParticleDensity=0.000000
SpellParticleNearClipPlane=2.000000
SpellParticleCastFilter=17
EnvironmentParticleDensity=0.000000
EnvironmentParticleNearClipPlane=2.000000
EnvironmentParticleCastFilter=24
ActorParticleDensity=0.000000
ActorParticleNearClipPlane=2.000000
ActorParticleCastFilter=17
UseD3DTextureCompression=TRUE
MipMapping=TRUE
TextureCache=TRUE
EnvSounds=0
WaterSwap=0
Bloom=0
TerrainTextureQuality=0
SkyUpdateInterval=60000
GraphicsMemoryMode=2
IgnoreFile1=1
Realism=0
ClipPlane=14
FogScale=2.800000
LODBias=0
Sky=0
LoD=1
HideCorpse=0
EnvironmentalDamage=1
ShadowClipPlane=0
Show3dTargetIndicator=1
FullscreenRefreshRate=0
FullscreenBitsPerPixel=32

 

[Akatora]

have you tried just loading eq without any other programs?

you can run them in windows and bypass the patcher by doing

D:\EQ\eqgame.exe patchme

after you create a desktop icon of the everquest.exe change it to that, or whatever your directory is... see if you can load them all that way and see if it still lags.. I am curious if its the 3rd party programs you are using to run them

 

[DriveCrash]

Drivecrash, I have added the nosound.txt file to the root directory. Does that apply similar to the above Sound=0 config in the .ini file?

The nosound.txt thing was older "kill switch" for eq sound, I don't know that it works anymore. I recommend adding the sound=0 to your ini as a precaution. It can't hurt.

You guys have been fantastic and very willing to help. I sincerely appreciate it.

You are most welcome!

- - - Updated - - -

ShowSpellEffects=1
TextureCache=TRUE
HideCorpse=0

I'd change those. No need for spell effects, and corpses piling up can tax the system.. however.. TextureCache I would absolutely shut down unless you are hitting a memory usage limit for your box.

Also, Last tips from me:
Set your character files to hide as much information as you can. Misses, faction, merc messages.. just anything that causes scrolling text boxes in EQ that you don't need to have. play with settings to make sure hiding text doesn't interfere with your macro, but any reduction to network packet information will be helpful.

Check your network for packet collision problems or any increase in latency. I don't think that could cause stutter framing in EQ, but who knows. The game has some very odd features left over in the code.

 

[Inkie]

...
Also, Last tips from me:
Set your character files to hide as much information as you can. Misses, faction, merc messages.. just anything that causes scrolling text boxes in EQ that you don't need to have. play with settings to make sure hiding text doesn't interfere with your macro, but any reduction to network packet information will be helpful.
...

I could be wrong, and it may have changed since I last paid attention to it, but I belive to impact network traffic you would also need the following in your eqclient.ini

serverfilter=1

I don't recall exactly what it said but the last time I tried to turn it on in-game it said something to the effect of "no longer does anything"

 

[Hihi]

You have your Journal turned off in game, and only have logging turned on on one toon if you're parsing? Alt J for journal window.