There are a few things relating to diesel combustion that determine engine torque. Basically getting fuel in quickly before top dead centre and burning it more or less completely - to create high combustion pressures at the most effective part of the stroke.
With old school fuel injection getting a lot of fuel in quickly creates loud diesel knock and creates excessive nitrous oxide emission, so our pumps are made for the necessary compromises.
Modern injection systems (e.g. unit injectors or common rail) can overcome this to suit common use - but for drag racing or tractor pulls then mechanical pumps modified for very high pressures and fast injection rule and modified common rail have so far been unable to compete - note this last comment is not relative to this thread, but it illustrates how much the injection rate and time relate to torque.
Tweaking our injection systems for more fuel has a limit to increasing torque because the extra fuel is injected later, i.e. injecting over a greater time, rather than more fuel in the same (or shorter) time.
The modern injection systems operate at higher pressure, which coupled with smaller nozzle holes gives finer atomisation of the fuel, which allows it to burn quicker and more thoroughly - this (better combustion) produces more torque (and power). It is also a problem if fuel quality is not too good.
In order to burn extra fuel, additional air is required and this is where turbos are required. VNT turbos can help with more air at lower engine rpm. Effectively turbos increase volumetric efficiency so a smaller displacement engine can produce the torque of a larger displacement normally aspirated engine - as others have said, off boost, the smaller displacement turbo engine would not produce as much torque.
It is not difficult or expensive to get 600+ Nm torque from a 4BD1T and it will not require internal changes (basically turbo, intercooler and injection pump tuning) and the engine will not still outlive the smaller alternatives spoken about in this thread.
In the later years of 300Tdi use in rovers the engines were produced by International in Brazil. The 2.8 was based on this. The 300 Tdi linage goes back to much older rover engines and some parts are still the same.
