The last chapter of Hacker's book is about a numeracy course that he designed and taught. I think it's a fine course that would admirably serve the needs of students in many majors. I have no problem with that. However, it's the wrong way to end this book. Hacker would not characterize his argument this way, but it's not as much a book about math as it is a book arguing for tracking on multiple levels.
On the college level, I think his point is mostly correct: Outside of the science and engineering disciplines, a lot of fields can and should modify their math requirements to be less abstract and more practical. It is still an argument for tracking to some extent--if you are a social science major who takes a lot of calculus and statistics you keep open the option of career paths that in which you're an early adopter of new tools, one who customizes them for the specific needs of your field. Every field has a few people like that. If you are a social science major who doesn't take those classes there are still plenty of things you can do in the workforce as a numerate person who understands a lot of things about people and societies, but you are closing certain doors. Still, I take his point: Most people just need to be numerate.
I still think that a broadly-educated person should attempt at least one real calculus or statistics class (as opposed to a "numeracy" class) if they are majoring in a subject that has highly quantitative sub-fields. But I see the point that most people will do fine in life if they don't take that class, or they try it but it doesn't work. I think most people are, on some level, fine with this sort of compromise, because college students are adults and choosing a major is all about turning away from most of the options in front of you to focus on a few closely-related possibilities.
But the other half of his argument is about high school, and here Americans tend to be much more ambivalent. For a number of reasons, I am OK with starting tracking in high school, partly because I think we unjustly devalue vocational education and partly because I see the consequences of pretending that we need everyone to be a STEM major. However, while I might personally be OK with it, this is a Big Issue. It's a significant departure from current practice, it involves educational issues that go well beyond math education (e.g. writing instruction for aspiring college students should be different from the writing instruction for people who plan to enter the workforce right after high school), and it hits on fundamental questions about democratic values. Emphasizing vocational education can do a lot for the middle class, but it will also lock a lot of people onto paths at a young age. There's a massive value judgment to be made there, and this book does not do justice to that issue. I can tell you that Hacker's argument would be a complete non-starter with most of my colleagues, and not just the mathematicians. Even the social scientists who would stand to benefit from greater emphasis on "numeracy" (which they might teach more effectively than most pure mathematicians) would have concerns about the equity issues in putting more people on non-STEM tracks.
Finally, while Hacker clearly respects math for its intellectual significance, I wish that he had shown that respect more openly, by taking the conversation one step beyond numeracy, at least at the college level. Let's grant that "college algebra" or a traditional statistics class should not be the gatekeeper courses for most non-STEM majors. Why should "numeracy" be the first and last encounter with math? Besides the fact that many people might benefit from a second course on statistics, what about math as a liberal arts discipline? The history of geometry encompasses agriculture, architecture, art (e.g. perspective drawing), and philosophy (via the Greeks' development of mathematics as a formal, logical, deductive discipline). Why not a general education course on that? Or a course on number theory (at an elementary level) and cryptography? I'm not talking about teaching humanities majors to do cryptography in any hard, formal, abstract sense, but why not have courses that try to give educated citizens a sense of what this tremendously important technology is really about, and why it ties into seemingly elementary ideas about arithmetic? How about formal logic courses taught one step down from what a math, philosophy, or computer science major might take, with the goal of getting the students to understand some aspect of the profound insights that Godel and Turing arrived at? I'd cheer for a school that required the non-STEM majors (and maybe even the STEM majors) to take a "numeracy" course plus one more math course from a menu that might include some of the ideas that I outlined here.