Kavanaugh's behavior indicates that he is more dedicated to protecting the privileges of whites, men and elites than other options might be. Which explains why they like him - nothing that's come out would make most Republicans dislike him, even if it were true.
But as for why they don't want to abandon him, I think there are several plausible explanations*:
1. Trump doesn't like to back down. This doesn't affect whether McConnell and others would
prefer to abandon him, but Trump sticking by him affects public opinion among the base and makes it more difficult for McConnell to pursue other paths.
2. While it will decrease in relevance as we move into October, they wanted to get someone on the court before the fall term began today. They already missed that deadline but he could still rule on the cases if he gets on reasonably soon. I don't know enough to know how soon it needs to be. Either way, if they think this is going to take weeks they'll be more inclined to just toss him overboard since this motivation will be moot.
3. Letting Kavanaugh go down for these allegations reflects on other Republicans who have similar, more numerous or more serious allegations against them, most prominently Donald Trump.
4. Ideological coherency matters to some Republicans, but even more important is raw power. And pushing through Kavanaugh and showing that you're powerless to stop them is part of what motivates them as well (which relates to why they like Kavanaugh and aren't bothered by the allegations, too). Or the short version: "to own the libs"
5. One of the smartest moves would be to withdraw Kavanaugh and nominate a woman or POC. A woman being part of the majority that overturns Roe v. Wade (explicitly or implicitly) would be an excellent troll move. The problem is that Trump isn't merely a cynical racist or sexist, nor is he as strategic as many of the other sincere racist misogynists in the party. He doesn't want to nominate a woman or a POC because he doesn't think they should be on the Supreme Court. Of course, this doesn't address why he doesn't withdraw Kavanaugh and nominate some other doughy-faced misogynist white man, but having fewer options for Plan B probably does make Plan A a little more appealing. (There may also be a legitimate argument that Trump's base would be dispirited by a woman or POC being nominated, even if he or she were a right-wing nutjob.)
Having specifically Kavanaugh on to rule on specific cases doesn't make much sense as a motivation, although I could see Trump thinking so because he's dumb. Kavanaugh's vote won't make the difference unless the other four conservatives on the Court are also going to rule that way. But if John Roberts is already willing to rule in favor of Trump on, say, self-pardons, then there isn't much reason to suspect that any other Republican hack wouldn't rule the same way. Particularly because Roberts is more concerned with the "legitimacy" of the Supreme Court than the other three conservatives.
Basically, if you have Roberts's vote, you'll very likely have the vote of anyone more conservative and more hacky than him, and people who are more conservative and hacky than Roberts are basically the only people who are under consideration in the first place.
Even if we presume that what they're concerned with is potentially replacing Ginsburg or Breyer and having 5 votes without needing Roberts's vote, you still need Thomas, Alito and Gorsuch. So just pick another Gorsuch or Alito! Kavanaugh isn't enough to guarantee a favorable ruling on his own.
*Some my own thoughts, but also informed by some of the links here:
Diversity! - Lawyers, Guns & Money (excluding Andrew Sullivan whose take was worthless, of course)
Why Do Republicans Still Support Brett Kavanaugh? - The Atlantic
Brett Kavanaugh is a man the right can get behind (relevant to point 4)