Since 1980, three waves of reformers have sought to transform American government. None succeeded. What went wrong? And what does it mean? In his new book Government's End: Why Washington Stopped Working --extensively revised since its first publication in 1994 as Demosclerosis -- National Journal senior writer Jonathan Rauch suggests an answer: The American public, having accepted limits on government's ability to change society, must now also accept equally exacting limits on society's ability to change government. An adapted excerpt follows.

The reformers in the '80s and '90s tried to pull the federal government to the right or the left. But, in the end, they did not remake government. It remade them.

To look back upon the 1980s and 1990s is to see what appears to be, at first blush, a period of quietude following the social and political storms of the 1960s and 1970s. The Reagan and Clinton years brought fiscal wars over deficits and culture wars over abortion and political correctness, but no Vietnam, no stagflation, no dogs and fire hoses in Alabama, and no chilling confrontations between democracy and totalitarianism. Intellectuals often complained that the Reagan period was complacent and vacuous, and that the Clinton years brought the abandonment of the activist spirit that once had energized American liberalism. The appearance, though, was partly deceiving. If American society was calmer after the 1970s, American government decidedly was not, for discontent with society had been displaced by discontent with government.

The era beginning in 1981 and ending, perhaps, in 1996 marks the most concentrated period of governmental reformism since the Progressives swept to power in Washington and in the cities nearly a century earlier. There was, however, a difference. The Progressives largely succeeded in breaking the old cronyist machines and replacing them with a class of professional administrators and a "clean government" ethic (with mixed effects, by the way). The reformers in the Reagan-to-Clinton years failed. They did not remake government; it remade them.

For the Progressives, the problem had been corruption and greed and the heavy hands of the bosses, who favored friends and shut out adversaries and thereby (in the view of the day) created political monopolies as damaging to the public good as were the great economic trusts. More openness, more access, and above all more professionalism were the answers. By 1981, when President Reagan took office, the Progressive formula had been turned on its head, although at that point few people realized the extent of the change. America's government was easily among the cleanest in the world or, indeed, in history. Endless safeguards of bureaucratic procedure and legal due process ensured that any decision that was deemed arbitrary or unfair could be challenged, first in administrative rulemaking, then in court, and finally in Congress. The civil service had been professionalized-and so, more recently and probably more importantly, had been the political class. It was now not only possible but common to be a full-time, professional lobbyist or political consultant.

And access? It was copious, redundant-so copious and redundant as to transform Washington itself into the site of a bidding war. With the old congressional seniority system weakened by the post-Watergate reformers of the 1970s, Congress now consisted of 535 individual entrepreneurs, each member chosen independently of party and president, each member a canny survivalist who could be asked to follow where the committee chairman or whip led but who could not be required to do so. For the (now) countless thousands of groups that professionally worked Washington, this meant that what you did not get from one member of Congress you could seek from another. The relationship worked the other way, too. When the politicians came calling on the lobbies for campaign money, as they did with growing brazenness, each group knew there were plenty of other lobbies, often competitors and adversaries, eager to help. If the Banking Committee chairman did not get what he wanted from the American Bankers Association, why, the credit union people or S&L people or insurance people or securities people were only too willing to step into the breach. The culture of government, by 1981, was honest and professional and astonishingly transparent; no one hid anything. But the economics of government, by then, was that of a piranha pool, with thousands of small but sharp-toothed and very strongly motivated actors determined not to be the loser at the end of the day. Every actor's activity, of course, drew in yet more actors. The Maryland state lottery once ran an ad campaign on the theme "If you don't play, you can't win." By the 1980s, Washington had become a kind of demented casino, whose slogan was "If you don't play, you can't win-but boy, can you lose!" Not surprisingly, everybody played.

The public, of course, was angry and disillusioned by 1981. The "trust in government" barometer had collapsed since the early 1960s. Confidence in government had been replaced with cynicism and suspicion. Among conservatives, a reform movement had arisen, in tandem with the change in the government itself. The movement was not progressivism so much as regressivism, but it was equipped with a powerful and sweeping critique of government and with a grand architecture of reform. Some liberals, too, dreamed of sweeping change. Ironically, however, although the liberal reformers and conservative reformers vied to pull Washington in opposite directions, they would soon discover they were trapped together like antagonistic prisoners thrown into the same cell. Both were mostly helpless.

If you view Washington's problems as superficial and transitory-the result of having elected this or that president, or of divided partisan control of government, or what have you-then the answer should be to elect some new leader or to consolidate power. If you think the problem is that the politicians are all the same, all empty suits wedded to the status quo, then the solution should be to elect some revolutionaries who will shake things up. If you think the problem is that reform in one direction simply goes the wrong way, then the right approach should be to try reform in the opposite direction.

As it turned out, the era of reform proved to be a uniquely useful natural laboratory for diagnosing government's condition, because many of the available permutations were tried. First the Republicans enjoyed effective control of both Congress and the White House, then the Democrats controlled both branches, and then control was divided. Far from electing empty suits, the voters on three occasions brought in strikingly fresh and energetic leaders, leaders who fervently believed in reform and who spared no effort to make it happen. And far from standing pat in moderation, the reform efforts lurched in two opposite directions. Reagan and House Speaker Newt Gingrich had pulled to the right, and had mostly failed (with a few important exceptions, such as Reagan's tax reform and Gingrich's welfare reform); Bill Clinton, with his sweeping health-care reform, had pulled to the left, but fared no better.

Yet, by the end of the 1990s, the reform era had subsided into exhaustion. The voters seemed to have given up, and there was no viable reform movement anywhere in sight. The battlefield was empty, the Bastille untaken, and the adversary little more than inconvenienced. In fact, the Washington establishment was fatter and happier than ever.

A Revolutionary's Blueprint
Of the reformers, none showed more energy and promise than Newt Gingrich. In hindsight (always the most discerning kind of vision), Gingrich seems to have been an overweening idealist who pushed his luck too far. But defeat appears inevitable only after the war. Gingrich did not enter office as House speaker without a plan. He explained it in January 1995, and it was not a stupid plan.

Gingrich was no newcomer. He went in with his eyes wide open. The power structure on Capitol Hill, he told The Washington Times as he assumed the speakership, had "ossified into a straitjacket. That is not partisan or ideological-these guys and their staffs had networks of power and networks of relationships and habits and things that they weren't going to break for a mere president. They'd ignored Nixon, Ford, Carter. They had blocked Reagan and beaten Bush." Moreover, "every time you mention something which ought to be shrunk or zeroed, twenty-five people who are making money off of it jump up to explain why it is a wonderful institution and they should continue to make money off it."

Gingrich's response, his battle plan, is instructive, because on paper it was plausible. First, he would mobilize his supporters, the fiery voters who had demolished Democrats and tossed out a reigning House speaker to put Gingrich and his reformers in charge. "The point we're going to make to people is, you'd better call your representative and tell them you want them to help pass the constitutional amendment to require a balanced budget-with a tax-increase limit. We're going to use every bully pulpit we have.... And we're going to tell every conservative group in the country and every group that wants smaller government, you'd better talk to your representatives." The intensity of the government reformers was high, Gingrich knew, so they could mobilize some of the same merciless spot pressure as the interest groups.

As for the Democrats, the 1994 election had thrown them into disarray. The president sounded chagrined, humbled, the wind knocked out of him. "I agree with much of what the electorate said yesterday," he said the day after the election. "They still believe that government is more often the problem than the solution. They don't want any party to be the party of government. They don't want the presumption to be that people in Washington know what's best.... I accept responsibility for not delivering to whatever extent it's my fault we haven't delivered." This humbled president would still wield a veto, but he would be presented with a stream of bills passed on the Republicans' terms in the wake of an election that had given them a mandate. If he refused to deal, he would risk seeming obstructive and deaf to the voters' demands. Anyway (said the Republicans to themselves), this was not a president who had shown a lot of backbone.

The lobbies, of course, could be counted upon to try to block or emasculate everything. Gingrich's response: swamp them. Attack so many programs at once that the Democrats and liberals and establishmentarians would have to choose the programs they wanted to save. The rest, the Republicans would knock off. The Democrats would have to "figure out which fights to stay and fight," Gingrich said. Gingrich was hoping to invert the usual Washington pattern, in which reformers were required to focus their energies on a few programs and let the rest of their agenda slide away. By attacking on a broad front, he would force the defenders to concentrate their fire. The Republicans would not get everything, but they would get a great deal.

Finally, Gingrich knew that at each stage of the process-House deliberations, Senate deliberations, House-Senate conferences, negotiations with the White House, presidential vetoes-he would lose bits and pieces of his agenda. A month or a year wouldn't be enough, a point he went out of his way to emphasize. Instead, he would start in 1995, running a flying wedge through the Washington power structure, and then come back again and again after that, widening the breach. There could be no "Mao-style revolution," he said. "I want to get to a dramatically smaller federal government. I think you do that one step at a time, but you insist on steps every year.... The reason I keep telling people to study FDR is if you take fourteen steps successfully you're a lot farther down the road than this guy next to you if he's trying to get all fourteen steps in one jump."

The trouble was, of course, that he never got to the second step. Why?

The Paradox Of Particulars
Voters in the polling booth vote for "change" in the abstract. But presidents and members of Congress can't. "In Congress, we don't get to vote on the abstraction," Republican Rep. Vin Weber of Minnesota told Time Magazine in 1992, shortly before retiring from office. "We have to vote for or against actual programs." That means confronting actual constituencies. Gingrich's hope to invert this equation foundered on the fact that in the case of any particular program or subsidy or perquisite of whatever sort, there is almost always far more energy on the defensive side than on the offensive side.

Say someone in Gingrich's position as a House leader hoped to reform or abolish a thousand programs. No one of those programs is essential to his effort. If he must, he can always drop twenty or thirty or even a hundred or two. There is no overwhelming incentive to go after any particular constituency. To the defender of the subsidy for left-handed screwdrivers, however, only one program matters: his own. He will spare no effort. For that defender, and for each of the others, it's life or death.

Gingrich understood this but thought he could count on his zealous Republicans to hold the line across a broad front. The discipline he was expecting, however, was superhuman. The temptation to help out this one group, or that one, was not Democratic or liberal; it was universal. After all, the clients understood that if one congressman would not help them, another might. Every congressman understood this, too. Why let someone else do the rescuing and take the credit? And every congressman also understood that every other congressman understood. And so, at every stage in the process, Democrats and Republicans demanded that this or that program be let off the hook. "I'd love to support you, Mr. Speaker, but I tell you, I am just taking a beating from those left-handed screwdriver people in my district-you've got to cut me a break." Facing this inevitable onslaught, Gingrich found that it was he and his reformers, not the Democrats or the liberals, who were swamped.

David Stockman, Reagan's reformist budget director, had run into the same problem, and had reacted with contempt for the gutless Republicans who were all for cutting government except the bits they wanted to save. Stockman, however, missed the point: Given the calculus of the game, the gutless Republicans were doing the only rational thing. The same sort of calculus had wrecked Clinton's health-care package in 1994. In fact, what was remarkable in 1995, arguably, was not how much the reform package was watered down in Gingrich's House (with significant program terminations shrinking by a factor of ten) but how large a tattered remnant actually survived.

Gingrich understood the importance of public mobilization. He counted on it to push his program past the Democrats and Clinton. In Gingrich's case, and also in Stockman's and Clinton's, the reformers depended on the public to rally around when political hackery began to prevail over the spirit of reform. And, sure enough, the public always did rally-but to the wrong side.

It turns out to be surprisingly easy for the protectors of programs to spook the public by screaming bloody murder. The public wants the government to be leaner, but not at the expense of students, farmers, bankers, workers, veterans, retirees, homeowners, artists, teachers, train riders, or cats and dogs. The people cannot abide the ghoulish shrieks and moans that are heard the moment the reformers' scalpel comes out. The same narrow focus and intense commitment that make lobbies so adept at defending themselves on Capitol Hill also make them good at alarming the public with "red alert" mailings and scary television ads (as with "Harry and Louise"). When all else fails, there is the old "Don't hurt our children" ploy. In 1993, when Congress managed to abolish the wool and mohair subsidy, the reformers were all the more courageous for having faced down pleas like the one from Nelda Corbell, whose parents raised mohair in Texas: "I am eight years old and I want to know why the government wants to take away our living." What kind of monster would hurt little Nelda?

Now and then, politicians manage to turn public opinion against a particular lobby, or at least they manage to exploit a change in public opinion, as the tobacco lobby found out. But usually they can't even do that. In his 1996 presidential campaign, when Bob Dole tried to mobilize public sentiment against the teachers' unions, he was judged quixotic. The public is nervous, often rightly, when politicians try to demonize some faction or other. Public nervousness makes the climate of opinion flammable; all that remains is to light a spark.

Rational Paranoia
In May 1981, President Reagan, on Stockman's advice, proposed a package of modest reductions in Social Security: reduced benefits for early retirees, a three-month delay in the cost-of-living adjustment, and so forth. The result was what Congressional Quarterly described as a "tempest in Congress." The Democrats until then had been helpless against Stockman, but they knew that this time he had stumbled onto vulnerable ground. The House Democratic caucus promptly and unanimously passed a resolution denouncing Reagan's "unconscionable breach of faith" and swearing not to "destroy the program or a generation of retirees." Democrats in the Senate promised to use "every rule in the book" to stop the proposal. "Democrats waged their assault with obvious glee," said Congressional Quarterly, and they kept waging it through the 1982 elections, when they gained twenty-six seats in the House and regained effective control there. Painting Reagan and the Republicans as scourges of Social Security received a good deal of the credit (the economic recession received most of the rest).

In 1995, Newt Gingrich's Republicans, responsibly and courageously, undertook to propose some modest but significant reforms of the Medicare program for the elderly. That the program's finances were in trouble, and that reductions would have to be made one way or another, were facts known to everybody in Washington, including President Clinton. He proposed reducing the growth of Medicare's costs from more than 4 percent a year for six years to 2.7 percent. The Republican plan, in not exactly sharp contrast, proposed reducing the growth path to 1.5 percent, with some larger structural reforms than Clinton preferred. In dollars, the difference between the plans was about 7 percent in the last year, 2002. But that was enough for the Democrats. Through the 1996 campaigns, they hammered the Republicans for "cutting" Medicare. "The Republicans are wrong to want to cut Medicare benefits," a voice-over intoned in one Democratic ad, as the faces of Bob Dole and Newt Gingrich danced on the screen. "And President Clinton is right to protect Medicare, right to defend our decision as a nation to do what's moral, good, and right by our elderly." The campaign became known as Mediscare, and it was accounted a great success. The public was quite willing to believe that Gingrich and his crew were out to gut Medicare. Despite their pleas of innocence, the Republicans never recovered.

In 1993, Bill Clinton proposed his health-care reform package. In 1994 came the "Harry and Louise" ads and plenty of others like them. Again, opponents had little trouble arousing public hostility to reform. So the trick works for both parties.

It works, you may say, because the public is ignorant and easily frightened. That explanation is right, to some extent. But it fails to give the public quite enough credit, because the public's suspicions were rational in each case. When the Gingrichites tried to make changes in Medicare, they plausibly argued that the (small) pain they were imposing on one group would be more than offset by the benefits to everybody from lower deficits, lower taxes, and a solvent Medicare program. But at that stage, the Democrats and the lobbies, acting as a swing vote, did exactly as the playbook suggests. They recast the debate as group versus group rather than as group versus nation. They stood on a box with a megaphone and warned: "Don't believe those Republicans! They're not going to give anything back to you once they've cut Medicare. They're financing tax cuts for the rich! They're just taking from you to give to their friends!"

Most Americans will sacrifice for a larger public good, but few will sacrifice for a competing group. The larger public loses interest in reducing Medicare, or in reducing anything else, if it believes that the only result will be to shift resources from one group to another. By kindling suspicions that the Republicans were acting in the interests of their favorite clients rather than of the nation as a whole, the Democrats and their allied lobbies had no trouble sinking the Republicans' Medicare deal. On health reform, the Republicans and the plan's other opponents used the same tactic against Clinton and the Democrats. "This plan doesn't mean more care at lower prices," they said. "It means poorer care for you and better care for other people, with huge new bureaucracies in the bargain."

Alas, this trick of kindling mistrust can almost always be used by somebody, because the charges, though overdrawn and often misleading, are usually plausible and partly true. The Republicans were trying to cut Medicare while also reducing taxes for better-off citizens. The Democrats were relying on bureaucratic controls to constrain choices for the middle class and expand health access for the poor. In 1981, the Reagan administration was trying to use Social Security reductions to help pay for upper-class tax cuts. In a democracy, parties do not get things done (or win elections) unless they favor their supporters, which means that the other side of any argument can always cry foul. And the voters' cynicism, which admittedly is often justified, makes them quick to believe charges that the system will double-cross them. The cynicism, of course, is self-fulfilling.

So here is the conundrum of collective political action. If you assume that everyone else will act in his rational self-interest, you have every reason to support politicians who put dollars or benefits or protections in your pocket, and little or no reason to support politicians who remove them. Although it is certainly possible to neutralize the opposition party and divide the lobbies and win the public's support, no sensible politician or voter ought to expect it to happen. Far more likely is the fate of the reformers of the 1980s and 1990s, who found themselves, after starting out well, suddenly staring at a coalition of opponents that comprised the opposition party, the lobbies, and the broad public. Against that array of forces, there is simply no hope. Reformers are crushed.

In the movie The African Queen, a famous scene has the protagonists' boat hopelessly stuck in a marsh-only a few yards, it turns out, from open water. Today's government is in a similar plight. Dissatisfaction ought, by rights, to open the path to comprehensive change. But it does not. The African Queen was lifted from the quagmire by the tide. But in the case of the American government, the boat cannot be lifted. The government is, of its nature, inseparable and inalienable from the million commitments it has made and the million interest groups it has spawned. They now form its environment. It cannot lift itself above them. With the replacement of Carter with Reagan, Bush with Clinton, and Clinton (for a while) with Gingrich, the restive electorate outside Washington showed that it could still radicalize politics, at least temporarily, and shake the very ground of the capital. Notwithstanding all the little gray groups and politicians and lobbyists and claques that occupy and ossify the government, the broad electorate proved more than able to coil itself and strike back. What was lacking in the system was not energy or leadership but the ability to focus reformist energy on any particular program of reform. Converting the electorate's shuddering waves of discontent into the hundreds or thousands of alterations to programs affecting specific groups is like converting earthquake energy into steam power: possible in theory but elusive in practice.

Borders Of The Jungle
In ideological terms, conservatives see government as properly a guarantor of individual rights, and possibly also as a watchman for the interests of enterprise. For 150 years or so, American government conformed largely to their vision. By today's standards, it was very small and very weak, and the country's many associations were of the voluntary, nonlobbying kind that were familiar to Alexis de Tocqueville in the 1830s.

Liberals see government as properly a solver of national problems, and possibly also as a builder of a more nearly ideal society. For thirty or forty or fifty years, beginning around the time of the New Deal, the liberals had their day: The government was ambitious, undertook all sorts of commitments to pensioners and veterans and students and consumers, and seemed often successful in meeting them. But with the growth of the programs came the dense jungle of modern Washington, with all its burrowing and flying and stinging creatures; and with the growing perception of failure-with farmers being paid not to grow food, the welfare culture expanding, the tax code becoming spaghetti, lawyers and lobbyists overrunning Washington, inflation, deficits, bureaucracies-came the backlash and the era of reform.

And now, at last, comes this, what you see around you: the perpetual stalemate of evolutionary equilibrium, in which the clients and the calculus of collective action will not allow the government to become much smaller or to reorganize its basic functions, while the taxpayers will not suffer it to grow much bigger. The borders of the jungle are more or less as they will be. From a distance, in macrocosm, the jungle seems an immovable mass, unchanging from year to year and impenetrably dense, whereas up very close, in microcosm, it is a constant turmoil of digging and scurrying and eating and mating. But it exists primarily to survive from year to year and to feed its clients. Its clients-we-draw sustenance from it but yield control.

In the end, it is not the conservative vision of government or the liberal vision that prevailed. It is no vision at all that prevailed. The client groups prevailed. And that is the end of government. To see the future, look around.

• Each week's National Journal cover story -- as well as other selected articles -- can be found in the NationaJournal.com News Archive. NationaJournal.com is a members-only Web site.
• Click here for more information about National Journal.

Need A Reprint Of This Article?
National Journal Group offers both print and electronic reprint services, as well as permissions for academic use, photocopying and republication. Click here to order, or call us at 877-394-7350.