I get Hacker's basic point: Most educated adults need a certain type of "numeracy" (in the language of John Allen Poulos), not advanced math. I get that. I do. And I get that PhD mathematicians, at least on the pure side of math (less so on the applied side) are usually not the best people to instill that. I even get that academic departments seeking to defend turf and bring in revenue are going to seek to expand courses that might not have much relevance to the actual needs of students.
I get all of that, but he starts going too far in some of his assertions. Yes, I'm going to nitpick, but I'm nitpicking because he's tossing out a lot of things largely to fill space. His basic point could have been made, explored, defended, and extended in a 3-part series of articles in a high-profile venue, and it probably would have gotten more attention while including fewer irrelevant and inaccurate asides.
On page 112 he says that mathematicians have done such a bad job of articulating the real value of their subject that 2013 only 730 US citizens got PhDs in math. There's much to say about how academic disciplines have failed to articulate the true value and beauty of their disciplines to the captive audiences in intro classes, but I don't know that we need to complain about "too few" PhDs. Elsewhere in the chapter he was complaining that intro math is often taught by low-paid and low-status part-timers. Well, over-production of graduate degrees is one of the key economic factors underpinning that model. The applied mathematicians tend to get industrial jobs, but the pure mathematicians are produced in greater numbers than the academic job market can absorb into full-time positions.
He also argues that many math courses for non-mathematicians should be taught by non-mathematicians. I will grant that a course on numeracy or "basic statistics for voters" might be better taught by a social scientist than a pure mathematician. And certainly science and engineering departments do tend to have their own mathematical methods courses on topics specific to their disciplinary needs. However, I think it would be a mistake to not have any of this done by mathematicians. An engineering department doesn't know in advance which of their students will go on to careers where they are end-users of signal processing software (where they might not need to know lots of theoretical detail) and which will go on to careers where they develop signal processing algorithms (where they will most assuredly need lots of theory). A physics department doesn't know in advance which of their students will go on to careers where they mostly characterize materials in the lab (where they will certainly need a lot of quantitative reasoning but might not need lots of eigenfunction expansions), which will become a professional theoretician (a rare path, but one that needs theory, by definition), and which will be the applied physicist who develops the algorithms for more efficient software in that materials characterization lab (where they will need theory). So a certain amount of exposure to the thought processes of mathematicians is an important aspect of a broad education.
As is so often the case with 200 page books sold to a broad audience, Hacker has a reasonable basic point but is over-reaching to fill space without diluting his message. I'm coming to hate this genre of books.