That is quite a strong statement. I took into account most of your arguments in my footnotes. For example, I noted how good Derek Rose and Russell Westbrook were in their box score +/- ratings, but that Net On/Off Rating were decreased by the overall strength of the team.
Secondly, I noted that players who had low Net On/Off ORtgs (box score +/-) were not really stars if their teams were so bad that it made their Net On/Off Rtgs jump (see Okafor).
Third, in 2016–2017, Jrue Holiday was a much better player than Tony Parker in every statistical category except assist/turnover ratio (Holiday’s 2.50 was still way above average among all NBA guards), received votes for the NBA All-Defensive Team. Offensive and Defensive Ratings, Box Score +/-, Win Shares, and Win Shares/48 min. don’t explain the gap between the two players.
Maybe we need a stat called plugability, the ability for a player to be put on almost any team and still excel. Isaiah Thomas for his entire career was nothing more than a high scoring bench player who couldn’t play crunch time minutes for a playoff team, then had two magical years in Boston. I don’t know if even Popovich would have been able to find a way to make Thomas work in his system, but it’s a certainty he wouldn’t have tolerated Thomas’ statements to the press and comments about other teammates. (BTW, San Antonio got to the Western Conference Finals that year, so the argument about weak starting lineups and strong benches not doing so well in the playoffs didn’t hold true in that instance.)
As far as context, I specifically said that On/Off Ratings can’t be used for reserves for the exact reason you stated. It’s always possible to find one reserve or starter who benefits from playing for a great team and ends up with a ridiculously high On/Off Rating. Did you not see my joke candidate for the 2019 MVP?
With regard to MVP-level players, don’t you find it interesting that Net On/Off Ratings, and the four context stats that make it up (+/- box score on court, +/- box score off court, Net On/Off ORtg, Net On/Off DRtg) were far better able to quantify LeBron James’ last two seasons (regardless of the fact that one team made the Finals and the other missed the playoffs) than traditional stats, PER Ratings or Defensive Rating? He had his best assist/turnover ratio since the 2013 title year, but he killed the flow of the Lakers offense just as much as Rajon Rondo did.
In addition, I explained that in cases where the On/Off Rating doesn’t match up with the eye test we do on a player, we can look at how they affect their team’s offense or defense. When Isaiah Thomas on-court presence raised Boston’s offensive rating so high (+116.7), which was better than the #1 rated Golden State Warriors (114.8). That’s a signal that a player is playing at an elite level at that end of the floor.
The same thing applied to Holiday on the defensive end, as the team dropped to #24 in defense when he was off the court. The proof of the pudding is that Holiday was All NBA-Defensive 1st Team the following year, and is once again being considered this year.
Finally, I think On/Off gives us a better starting point to then look at other stats for context and to prove or disprove a theory about a player’s value. For every outlier like Robert Covington, there are ten empty stat guys in the league who are massively overpaid.
It’s funny that you can wax poetic about Doncic (“only one of X rookies who ever did Y!”) without looking at a player like Jrue Holiday and wondering how on earth does a guy complete a season that has only been accomplished by seven other players in NBA history and not make an All-NBA team.
