The Language of Mathematics: Making the Invisible Visible by Keith Devlin
The Mathematical Universe by William Dunham
Mathematically Speaking: a Dictionary of Quotations by C. C. Gaither
Why People Believe Weird Things by Michael Shermer
(one of the best books I've read on the difference between science and pseudo-science. Wonderful!)
The Story of e by Eli Maor
Clear and beautiful. Primarily for the layman, but it also contains material for the mathematically sophisticated reader.
Schrödinger's Cat by John Gribbins
A layman's explanation of the strangeness of quantum mechanics. Sometimes he gets carried away, but it's a good introduction to the philosophical side of the science.
The Character of Physical Law by Richard Feynman
Quirky Feynman, but written for the layman. Interesting insights into general physics.
The Tao of Physics
A mystical view of particle physics. Although there are a few errors and hyperbole, it is much better than its competitor, The Dancing Woo Li Masters (which is full of mistakes and misunderstandings of basic physics)
New Ideas in Modern Physics
Well written compendium of all branches of physics for the science-educated layman .
The Making of the Atomic Bomb by Richard Rhodes
This is the history as well as the science (a little above layman-level) of the Manhattan Project and the race in Germany and elsewhere to develop the atomic bomb. It deservedly won the Pulitzer prize. I couldn't put it down.
Flatland by Edwin A. Abbot
I read this classic when I was in high school and it forever changed the way I look at the universe. It is written in an old style that at first might seem cumbersome, but if you stick with it you will see its artistry.
The Fourth Dimension by Rudy Rucker
This book is whimsical. It gives you various exercises to try to get you to "visualize" a fourth dimension. If you plan to read this, read Flatland first.
Subtle is the Lord by Abraham Pais
This biography of Einstein is not only about the man but also about his science. It is rigorous and not for the faint-hearted, although it can be read without delving into the scientific details. This book is considered to be the best ever written on Einstein.