There is no way except disconnecting to get out of the spectate mode.
And why does my player body have to die when getting into spectate mode?

Because that is the official implementation. :)
I looked at the official code and what they do is identical to how I do it.

The way you are supposed to "unspectate" is by respawning.

/* My code */
[ConsoleCommand(CSpectateUserCmd)]
private void PlayerAdministrationSpectateUserCallback(ConsoleSystem.Arg aArg)
{
	LogDebug("PlayerAdministrationSpectateUserCallback was called");
	BasePlayer player = aArg.Player();
	ulong targetId;

	if (!VerifyPermission(ref player, CPermSpectate, true) || !GetTargetFromArg(ref aArg, out targetId))
		return;

	if (!player.IsDead())
		player.DieInstantly();

	player.StartSpectating();
	player.UpdateSpectateTarget(targetId.ToString());
	LogInfo($"{player.displayName}: Started spectating user ID {targetId}");
	BuildUI(player, UiPage.PlayerPage, targetId.ToString());
}

/* Official code */
// ConVar.Global
[ServerVar]
public static void spectate(ConsoleSystem.Arg args)
{
	global::BasePlayer basePlayer = args.Player();
	if (!basePlayer)
	{
		return;
	}
	if (!basePlayer.IsDead())
	{
		basePlayer.DieInstantly();
	}
	string @string = args.GetString(0, string.Empty);
	if (basePlayer.IsDead())
	{
		basePlayer.StartSpectating();
		basePlayer.UpdateSpectateTarget(@string);
	}
}

Ok. So no way to end spectate mode without dying cause Facepunch wrote it that way, right?

Your player has to die, as there is no body for the spectating admin. (Would be strange, no?)
I would also prefer not to change how things are done, and I am also not able to make a "unspectate" because that would mean I have to make a respawn that puts your player back into it's previous state (Not interested in doing that, the plugin is already heavy enough in memory)

Ok. Thank you.