Got this exact same crash again tonight. Anyone had any progress please?
Linux server crashing with caught fatal signal
- 1
- 2
Disclaimer: Use this at your own risk.
Still too early to tell if what I have done has permanently fixed this problem or not, but for the last week the servers have been crashing on restart every single day.
During this month it's been particularly bad for some reason, they crash on restart everytime the entities get over 220,000. This has been going on intermittently for over a year.
This morning I changed the vm.max_map_count from default of 65536, to 2262144
Follow the instructions below to change the setting.
sudo sysctl -w vm.max_map_count=2262144
Now open the sysctl.conf file.
sudo nano -w /etc/sysctl.conf
Add the following line to the end of the sysctl.conf so the number stays changed when you reboot the hardware.
vm.max_map_count=2262144
So far so good, all 5 servers have stopped crashing.
I'm not 100% sure yet if this is a fix, but so far it seems to be working. The servers have been up for over an hour, normally they would crash after restart within 3 - 15 minutes.
Still too early to tell if what I have done has permanently fixed this problem or not, but for the last week the servers have been crashing on restart every single day.
During this month it's been particularly bad for some reason, they crash on restart everytime the entities get over 220,000. This has been going on intermittently for over a year.
This morning I changed the vm.max_map_count from default of 65536, to 2262144
Follow the instructions below to change the setting.
sudo sysctl -w vm.max_map_count=2262144
Now open the sysctl.conf file.
sudo nano -w /etc/sysctl.conf
Add the following line to the end of the sysctl.conf so the number stays changed when you reboot the hardware.
vm.max_map_count=2262144
So far so good, all 5 servers have stopped crashing.
I'm not 100% sure yet if this is a fix, but so far it seems to be working. The servers have been up for over an hour, normally they would crash after restart within 3 - 15 minutes.
I've also been having this problem on all 4 of my Rust servers for the last 4-6 weeks.
Sometimes they reboot fine, sometimes they crash with a "Caught fatal signal - signo:11" error of some sort.
Sometimes they crash 5-10 times in a row, then finally start up ok.
The specific details shown below the signal message can vary quite a bit from crash to crash.
I'm running on Debian 11/kernel 5.10.0-11 (latest as of this post), and I run all my servers in docker containers.
I tried RustNRelax's suggestion, but it doesn't seem to have worked (for me, anyway) unfortunately.
I also tried removing the Discord DLL from my build to rule it out - I still got the signal 11 error.
Any help/info/suggestions appreciated. I've included a sample error below.
Feb 2 03:01:28 hv2 dev-rust[2576]: Caught fatal signal - signo:11 code:2 errno:0 addr:0x7f6e94f1d8a0
Feb 2 03:01:28 hv2 dev-rust[2576]: Obtained 11 stack frames.
Feb 2 03:01:28 hv2 dev-rust[2576]: #0 0x007f6f93962140 in funlockfile
Feb 2 03:01:28 hv2 dev-rust[2576]: #1 0x007f6f943c44c6 in GarbageCollectSharedAssets(bool, bool)
Feb 2 03:01:28 hv2 dev-rust[2576]: #2 0x007f6f943d9e21 in UnloadUnusedAssetsOperation::IntegrateMainThread()
Feb 2 03:01:28 hv2 dev-rust[2576]: #3 0x007f6f943d914b in PreloadManager::UpdatePreloadingSingleStep(PreloadManager::UpdatePreloadingFlags, int)
Feb 2 03:01:28 hv2 dev-rust[2576]: #4 0x007f6f943d98ea in PreloadManager::UpdatePreloading()
Feb 2 03:01:28 hv2 dev-rust[2576]: #5 0x007f6f943c685e in ExecutePlayerLoop(NativePlayerLoopSystem*)
Feb 2 03:01:28 hv2 dev-rust[2576]: #6 0x007f6f943c68a1 in ExecutePlayerLoop(NativePlayerLoopSystem*)
Feb 2 03:01:28 hv2 dev-rust[2576]: #7 0x007f6f943c6dda in PlayerLoop()
Feb 2 03:01:28 hv2 dev-rust[2576]: #8 0x007f6f945306de in PlayerMain(int, char**)
Feb 2 03:01:28 hv2 dev-rust[2576]: #9 0x007f6f937afd0a in __libc_start_main
Feb 2 03:01:28 hv2 dev-rust[2576]: #10 0x0055fd93c00699 in _startMerged post
Please ignore my previous post - it turned out to be faulty memory in my server causing the signal 11 error.
After replacing the RAM, I've no further issues. Sorry to add to any confusion here.
Unfortunately this issue is still ongoing for me aswell,
Caught fatal signal - signo:11 code:1 errno:0 addr:0x6c
Obtained 16 stack frames.
#0 0x007fb914690730 in funlockfile
#1 0x007fb915caa551 in physx::PxsNphaseImplementationContext::unregisterContactManager(physx::PxsContactManager*)
#2 0x007fb915c4c655 in physx::S*:*:*:*:*:*:*:*Scen*:*:*:*:*:*:*:*finishBroadPhaseStage2(unsigned int)
#3 0x007fb915c4c7e8 in physx::S*:*:*:*:*:*:*:*Scen*:*:*:*:*:*:*:*updateCCDSinglePassStage3(physx::PxBaseTask*)
#4 0x007fb915c1cd24 in physx::Cm::Task::run()
#5 0x007fb9151cfd4a in PhysxJobFunc(physx::PxBaseTask*)
#6 0x007fb9150ec949 in JobQueu*:*:*:*:*:*:*:*Exec(JobInfo*, long long, int)
#7 0x007fb9150ecad4 in JobQueu*:*:*:*:*:*:*:*ExecuteOneJob()
#8 0x007fb9151f7890 in CompletionTask::ProcessAllPhysicsTasks()
#9 0x007fb9151cb627 in PhysicsManager::Simulate(int, float)
#10 0x007fb91513885e in ExecutePlayerLoop(NativePlayerLoopSystem*)
#11 0x007fb9151388a1 in ExecutePlayerLoop(NativePlayerLoopSystem*)
#12 0x007fb915138dda in PlayerLoop()
#13 0x007fb9152a26de in PlayerMain(int, char**)
#14 0x007fb9144e109b in __libc_start_main
#15 0x005577f6acc699 in _start
Did anyone find a fix for this? Been getting this error this past week with random restarts, restart loops,
Caught fatal signal - signo:11 code:1 errno:0 addr:0x7da4e0eb99d8
Obtained 24 stack frames.
#0 0x007fa799971980 in funlockfile
#1 0x007fa79a84f508 in void st*:*:*:*:*:*:*:*introsort_loop<Object, long, gnu_cxx::ops::_Iter_comp_iter<CompareInstanceID> >(Object, Object*, long, gnu_cxx::ops::_Iter_comp_iter<CompareInstanceID>)
#2 0x007fa79a84e7e4 in Object::FindObjectsOfType(Unity::Type const, dynamic_array<Object, 0ul>&, bool)
#3 0x007fa79ab76b2c in Scripting::FindObjectsOfType(ScriptingSystemTypeObjectPtr, Scripting::FindMode)
#4 0x007fa79ac291cd in Object_CUSTOM_FindObjectsOfType(ScriptingBackendNativeObjectPtrOpaque)
#5 0x00000041a95f7d in (wrapper managed-to-native) UnityEngine.Object:FindObjectsOfType (System.Type)
#6 0x00000041ce6db0 in SpawnHandler:EnforceLimits (bool)
#7 0x00000041c0d7ec in ServerMgr:Initialize (bool,string,bool,bool)
#8 0x00000041b873c4 in Bootstrap/<StartServer>d19:MoveNext ()
#9 0x0000004156439a in UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)
#10 0x007fa798a40505 in mono_print_method_from_ip
#11 0x007fa798bafa48 in mono_perfcounter_foreach
#12 0x007fa798bb08f5 in mono_runtime_invoke
#13 0x007fa79ab79dd2 in scripting_method_invoke(ScriptingMethodPtr, ScriptingObjectPtr, ScriptingArguments&, ScriptingExceptionPtr, bool)
#14 0x007fa79ab781aa in ScriptingInvocation::Invoke(ScriptingExceptionPtr, bool)
#15 0x007fa79ab5b72a in Coroutin*:*:*:*:*:*:*:*InvokeMoveNext(ScriptingExceptionPtr)
#16 0x007fa79ab5b365 in Coroutin*:*:*:*:*:*:*:*Run(bool)
#17 0x007fa79a9075b2 in DelayedCallManager::Update(int)
#18 0x007fa79aa67c2e in ExecutePlayerLoop(NativePlayerLoopSystem)
#19 0x007fa79aa67c71 in ExecutePlayerLoop(NativePlayerLoopSystem)
#20 0x007fa79aa6818a in PlayerLoop()
#21 0x007fa79abd054e in PlayerMain(int, char**)
#22 0x007fa79958fbf7 in libc_start_main
#23 0x0055d32546f699 in _startand even a wipe due to this error.
- 1
- 2