This isn't really a math textbook, but math is an extremely important part of physics. Physics textbooks usually at least attempt to include math support for key ideas, reviewing e.g. how to do a cross product. The problem with this is that this topical review tends to be scattered throughout the text or collected in an appendix that students rarely find when they most need it (either way).
I don't really like either of these solutions. My own solution is eventually going to be to write a short lecture-note style math textbook that contains just precisely what is needed in order to really get going with physics at least through the undergraduate level, including stuff needed in the introductory classes one takes as a freshman. Most mathematical physics or physical mathematics books concentrate on differential equations or really abstract stuff like group theory. Most intro physics students struggle, on the other hand, with simple things like decomposing vectors into components and adding them componentwise, with the quadratic formula, with complex numbers, with simple calculus techniques. Until these things are mastered, differential equations are just a cruel joke.
Math texts tend to be useless for this kind of thing, alas. One would need three or four of them - one for vectors, one for calculus, one for algebra, one for complex numbers. It is rare to find a single book that treats all of this and does so simply and without giving the student a dozen examples or exercises per equation or relation covered in the book. What is needed is a comprehensive review of material that is shallow and fast enough to let a student quickly recall it if they've seen it before well enough to use, yet deep and complete enough that they can get to where they can work with the math even if they have not had a full course in it, or if they can't remember three words about e.g. complex variables from the two weeks three years ago when they covered them.
In the meantime (until I complete this fairly monumental process of splitting off a whole other book on intro math for physics) I'm putting a math review chapter first in the book, right here where you are reading these words. I recommend skimming it to learn what it contains, then making a slightly slower pass to review it, then go ahead and move on the the physics and come back anytime you are stumped by not remembering how to integrate something like (for example):
Here are some of the things you should be able to find help for in this chapter:
Integers, real numbers, complex numbers, prime numbers, important numbers, the algebraic representation of numbers. Physics is all about numbers.
Algebra is the symbolic manipulation of numbers according to certain rules to (for example) solve for a particular desired physical quantity in terms of others. We also review various well-known functions and certain expansions.
Cartesian, Cylindrical and Spherical coordinate systems in 2 and 3 dimensions, vectors, vector addition, subtraction, inner (dot) product of vectors, outer (cross) product of vectors.
There is a beautiful relationship between the complex numbers and trig functions such as sine, cosine and tangent. This relationship is encoded in the ``complex exponential'' , which turns out to be a very important and useful relationship. We review this in a way that hopefully will make working with these complex numbers and trig functions both easy.
We quickly review what differentiation is, and then present, sometimes with a quick proof, a table of derivatives of functions that you should know to make learning physics at this level straightforward.
Integration is basically antidifferentiation or summation. Since many physical relations involve summing, or integrating, over extended distributions of mass, of charge, of current, of fields, we present a table of integrals (some of them worked out for you in detail so you can see how it goes).