Firstly: The laws of physics allow for a 70-mile high tower, even if it takes an incredible amount of material. Such a project would be massive in scale, and would use about 100 times as much concrete as Hoover Dam. It'd instantly be a wonder of the world comparable perhaps to the Pyramids of Giza. It's primarily not done for this reason, because it represents an incredible cost that only a few countries are capable of.
Secondly: Even if a large tower is built, the vehicle would still need at least 14,000 miles to safely accelerate at 3G. So unless you want to fill the vehicle with the necessary engines/propellant to achieve orbital speed in the upper atmosphere, you'd need a 14,000 mile long roof or tethered satellite rail gun to launch the vehicle. For context, the earth's circumference is about 24,000 miles and America's transcontinental railroad (two steel rails, no maglev) is only 2,200 miles.
Thirdly: Solutions to the above problem (say engines powered by beamed power, or just better engines) lead to atmospheric flight becoming easier which confers more benefits at a lower cost.
That said, these problems do not exist on the Moon or other planets which lack atmospheres and have low gravity.