Thanks for your answer. A few question though:
1) I didn't understood your first criterion though. What means the "H2H of the field against itself"? And if you analyze the H2H of a player against the top player AND against the field, is it not the same to look at his achievements?
Within modern times, the best way that I've personally done this is by looking at the "power band" of the top players using ATP points as the indicator. Now, I know that the total available ATP points has changed over time, so I really only use the percentage of the total ATP points held as that indicator.
The "power band" of the top players in a field will see a steady and consistent drop off in the distribution of ATP points. A weak field will see a much more drastic drop-off.
And just to show people that I really look deeply into this stuff, here is the beginning of one such analysis I did for the field from 2004 to 2009. This shows the raw power bands of the Top 100 for those years:
2) How do you use memory or video or a close match to compare two players? If I left out achievements, which are only a product of context and say nothing of playing level, I personally find it hard to rank Safin, Federer, Nadal and Verdasco from my memory of their AO SF. They all played at a very high level, both matches where very close.
I agree, it gets tougher here. Especially if individual performance statistics are sparsely available. Things become more subjective. I'd say video is much more effective than memory. But the ending score alone isn't indicative of match quality. Two crappy performances will yield similar lines to two high-quality performances.
I personally have a pretty good memory of high-quality matches between Federer and Nadal. As we as between Federer/Nadal and a few other players. But things can get foggy.
But I think in many situations (especially talking about 60s, 70s, and even some 80s) video evidence is absolutely necessary to see what we're talking about.
3) I assume you have analyzed the H2H of Roche against the top player and against the field, and the field of the late 60's with itself, an that you have done the same things respectively for Hewitt, Nalbandian, Roddick and Davydenko. Do you care to elaborate, and provide to us the demonstration?
I didn't go into those guys individually and looking at them specifically. I stopped at the field analysis from Fed's years of dominance.
4) How do you control the "crush" factor? Imagine if someone grew up watching attacking all court tennis, it's likely that he will be more impressed by net skills and touch than by a ruthless retriever. Or the opposite (we see that a lot these day, with all the djokoisthemegaiestforcethread). Is one style superior to the other?
I do agree that some can get caught up in the "romanticness" of what they've been exposed to. I personally don't think any style is "superior" to any other.
Within an era, I'm sure there is a style that is best-suited for success... but I don't think any particular style is necessarily best-suited for all eras.
5) Looking at a close matches allow to assess great technical skills. But how do you measure mental, physical and emotional skills, which are just as important, if no more, than the technical ones? These skills that make Ferrer what he is, and Nalbandian the opposite?
Awesome question. To me, this becomes a question of consistency. A player who has superior intangible skills (mental, emotional, fitness, consistency) will perform consistently better over time than someone who has lesser intangibles.
The examples you bring up are perfect. Ferrer, while he does not necessarily have the level of tangible playing skills that his rivals has... he definitely scores high on the intangible skills area. And this has allowed him to outperform people like Nalbandian. Nalbandian, like many players, has flashes of brilliance... but failed more often than not over time.
I'd also point out that even though many people rip Ferrer... he probably is the most balanced player in the ATP aside from Djokovic. I'd go so far to say that Ferrer is one of the most balanced and consistent tennis players ever.