Well that has eliminated all the accessories and belt as source of noise, with belt removed you are running only on battery and that rougher running may be showing up a voltage/battery related issue affecting the idle, alternator runs the vehicle once started, battery optional.
Not relevant for now just hunt the noise down.
After checking all the items short of pulling the head off check compression on cylinders (not had mine long enough to do this myself) .
I think I read it is done via the glow plug holes for 1 - 4. No idea how to check 5 (will let you do the google search on that one)
You will have to decide down the track whether to live with it i.e. wait for something to go bang or fail to really point you in the right direction or pull things apart.
Also try and think back to when you noticed it, was it a sudden change or something that may have been creeping up on you?
Take notice of things like amount of blowby coming out of rocker cover in case of broken rings/piston (hence the compression check)
Remove sump and check internals that way before taking head off is another thing. And loctite oil pump bolt while you're at it if you go that route.
Anyway just go ahead and check doughnut bolts for tightness and flex plate for cracks as planned b4 going further.
A bit of broken valve can be bouncing around although that tends to be a more rattly noise but still in a rat tat tat sequence as piston comes up and down.
There are just so many variables but sometimes you find it quick sometimes not, just patience and work through a process of elimination.
I'm just waiting for Discorevy or someone else who works on these in a pro capacity to chime in, I am not a qualified mechanic just a diy with a bit of experience. And new to these donks but an engine is an engine you just learn their behaviour and idiosyncrosies.

