Annoying is the right word here, because the Cheating Bastard build is far from unbeatable. There are opponents who almost always lose against it and opponents who trample it but there isn't much middle ground where the fights get really interesting. It's still slightly below the annoyance level of the Rage dodgers who are a purely RNG build (rolls are on your side - you win, they are against you - you lose) but that's mostly because they have bigger skill selection which allows for somewhat higher flexibility.