I've seen rejections of the Axiom of Infinity. This is called finitism. Some ultrafinitists even add the negation of the Axiom of Infinity. Definitely doable.
I've seen rejections of the Axiom of Choice. That would just mean you choose to work in ZF and not ZFC, which is controversial but not unheard of.
I could imagine rejecting the Existence axiom. Then all of your mathematics would be in one big conditional (where $\exists x \ x=x\implies\phi$) for any correct statement derivable in your axiom system (with the axiom $\exists x\ x=x$ removed). I can see a little formalist sparkle in this - you can manipulate all the symbols out there, and your manipulations would result in some truth if and only if there's some mathematical object out there you can play with. Maybe there isn't! Maybe there is! But this cuts down on our need to assume something exists and means we can do all of mathematics within logic (rather than making non-logical ontological claims). Does this have a name?
Or, even more bizarrely; do some people reject Comprehension, or Specification, or Pairing or whatever? Is this just obviously regressive thinking? Can we derive much much less?
I'm most familiar with ZF(C) and Peano Arithmetic, so I'd appreciate examples to do with those (maybe there's some niche axiom system and an even more niche modification where you reject 17 out of 199 axioms, but this isn't really what I'm looking for).