So let's take a step back and start from the beginning. That TAP guideline for exceptions say that an exception should only be raised as response to a usage error that should never really happen. This leads to code looking like the hybrid version you find here. One public method validating arguments that then calls a private method that do all the asynchronous stuff.
Since every project I've been on in the last five years all have had a component that used a different asynchronous pattern that was moved over to TAP, splitting up existing methods in two methods never felt like the right thing since all these methods were only used internally in the project. And it never felt important to enforce the TAP guideline since it was always some internal API that was changed.
I have actually been happy that I didn't need to make up my mind about how I felt about the need for two different methods all the time. Then this discussion at work happened and it dawned on me. While it is nice to throw argument exceptions before task is started as a way to distinguish usage errors from execution failures, the calling code rarely knows the difference.
If you are implementing a TAP based interface it is probably because you expect the caller to be using the async/await keywords when calling your code. And using those keywords the caller of your API does not know nor care how your method fails.
So it turns out that the way argument exceptions are handed over to the caller only matters when the caller is not using the most appropriate asynchronous pattern and I think I can live with the fact that I don't follow the TAP guideline to the letter since the caller should handle both types of failures the same way regardless of what I do.
When I was looking up the TAP guideline link i noticed that the language used on MSDN today is (to me) less prescribing than the language I encountered internally at Microsoft so maybe nobody is actually implementing TAP in this slightly more cumbersome way.
And yet I can't make up my mind 100% since I do find it appealing that certain errors doesn't even start a task before they fail. Very convenient in situations where multiple tasks are started in parallel in order to later evaluate the result.