Only Microsoft could tell you that for sure, or what exactly is the contract behind that function, but Microsoft basically says you should get away from this function.

The RtlGenRandom function is available for use in the operating systems specified in the Requirements section. It may be altered or unavailable in subsequent versions. Instead, use the CryptGenRandom function

Anyway, if you disassemble it, you find the

ProcessPrng -> AesRNGState_generate -> SymCryptRngAesGenerate

function call chains and there are also EnterCriticalSection calls along the way, so I'd "imagine" it's thread safe.

PS: the last SymCryptRngAesGenerate function is open source.

See also this crypto/rand: Currently using deprecated API for random number generation on Windows github discussion about the subject.