Myonie, no one is saying that it's ONLY because of his jokes about Jews and LGBTQ+ people that he no longer deserves a platform. It's precisely because of what you said--that he had been doing this for decades, using jokes that perpetuate stereotypes and harm to most groups of people, that there in no longer a reason to back this kind of comedy. The amount of time that he allotted to mock trans people in this new special has an effect on a society that is in flux with regards to whether or not it even wants to extend basic human rights to trans people. The average Black trans woman doesn't live past the age of 35--they have a higher likelihood of being murdered than literally any other group. It's just not funny to mock people to this extent who are struggling so deeply.
I'll clue you in also that you asking about the "fragility" of these groups is A) a testament to your own bigotry, not that this is likely to concern you based on your audacity to type this out in this way, and B) ridiculous because all human life is fragile. Yea, trans people have a right to speak up because THEIR lives are attacked and diminished and ended earlier than most lives should against their will because of hateful murder and systemic oppression. If you happened to be part of a group that was more vulnerable than others, you might have the humility to understand that you aren't immortal either, that even your life is precious and fragile, and that you would probably do something about it if you felt the weight of an entire society hating you and wanting you dead.
And calm down, no one is saying that no one shouldn't make jokes about women. But no, people shouldn't make jokes about *violence* against women--including rape. If you were clued into the stats about domestic violence and femicide, you'd understand that again, based on the sheer magnitude of this phenomenon (but also some basic principles of human compassion where you don't actually need someone else's pain as your punchline, unless you're a sociopath or an abuser or a sadist) renders any mockery of its victims as tacit encouragement and/or belittling of the issue and/or blaming of the victim. It's not helpful.
Yes, he's been doing this for a long time. Yes, our society has long had a much higher tolerance for bigotry than it does now. He made *this* special in 2021, and society -- not everyone, as evidenced by you, obviously-- is largely moving on past the idea that harming and publicly mocking people for things they can't help when it causes real pain (and affects legislation on people's bodies based on how it's normal to regard or dehumanize people) is "funny" and thus impenetrable and inscrutable. If this is the hill you'd like to die on, please boo go right ahead. But you don't have any moral highground. You're defending bullies and the freedom for bullies to be celebrated and elevated above all the millions of people who can do what they do, possibly even better, without causing harm.
By the way, I think about addiction to mean humor as tantamount to addiction to painful sex. It's an expression of your own self-loathing and your own inability to find joy or grace or laughter in things that don't involve harm--including to yourself. Even you deserve better, babe. It takes practice but it can be overcome if you can be strong enough to try :)