On a Virginia plain in December 1862, a group of Union soldiers was backed up toward a canal ditch under heavy mortar fire. It was the Battle of Fredericksburg, one of the largest of the Civil War. As the Confederate mortar rounds fell into the narrow canal and exploded, Union Lieutenant Colonel Edward Roberts noticed something curious: columns of water, many feet high. The behavior of the water sparked an idea. Roberts’ intuition was that explosive force could aid in drilling for oil, which had emerged as an industry in the U.S. shortly before the outbreak of the war.
Roberts left the army, and by 1865, the year of the war’s conclusion, had formed the Roberts Petroleum Torpedo Company. A “petroleum torpedo” was Roberts’ term for what today we call “fracking” – short for hydraulic fracturing. In the last 15 years, fracking has enabled the U.S. to return to its original perch as the largest producer of hydrocarbons in the world, after a half century of losing that battle to nations with easily-accessible reserves.
The basic principles behind fracking haven’t changed; by fracturing rocks, we can release trapped oil and natural gas by creating more pathways for them to drain into an oil well. Roberts’ torpedo, when lowered into oil wells, would explode sideways, fracturing the oil strata around the hole. He would then fill the wellbore with water in a process he dubbed “fluid tamping.” This process boosted oil production by as much as 1,200 percent from some wells within a week of being shot.
Roberts’ high fees resulted in black market alternatives developing. Groups of workers dubbed “moonlighters” and “torpedoists” would conduct explosive drilling at night to avoid detection. This enraged Roberts, who hired Pinkerton private detectives and an army of lawyers to protect his patent and its commercialization.
Accidents were commonplace. John Wilkes Booth and his partners destroyed their company’s Pennsylvania oil well when trying to use explosives to speed up production. Frustrated and angry about both the failure of his business and the South’s defeat in the Civil War, Booth assassinated President Abraham Lincoln. By 1868, the primary fracking technique was to drop a “go devil” weighted detonating device down a wire line into a well, in order to set off a canister of nitroglycerin (dynamite). Accidental detonations often set workers on fire, and resulted in many deaths. Over time, new techniques emerged. A large underground machine gun became a popular way to unlock oil in the 1930s. Later, a Swiss engineer invented a bazooka as part of a secret U.S. Army program and began firing it in Texas oil fields.
What we think of as fracking today emerged in 1949 outside Duncan, Oklahoma with the first commercial application of hydraulic fracturing by Haliburton and Stanolid engineers. At first, liquids such as crude oil were injected into poorly performing wells to increase their flow rate. Engineers improved the technique over time through countless experiments and testing of new formulas of fracking material.
More adventurous methods of fracking were also tried, including using nuclear explosives! As part of the U.S. Atomic Energy Commission’s Project Plowshare, nuclear explosives were tested at a site in New Mexico in 1967. The blast produced a rubble chimney as high as a 35-story building and nuclear-enabled fracking was never attempted again.
Despite technical advances, fracking was slow to be adopted. The technique was too expensive and added too much time to the drilling schedule. This would begin to change in the 1970s, when the U.S. received a shock that would change the oil world forever.
The U.S. was the undisputed oil superpower from the mid-1940’s through 1970. During World War II, the U.S. provided over 85% of the Allies’ oil supply and five of the “seven sister” companies that dominated the global petroleum industry were American. Meanwhile, a lack of the oil supply needed to power their industry and military motivated the Japanese invasion of the Dutch East Indies and Germany’s invasion of Ukraine.
When foreign countries control your energy supply, there are real limitations on your sovereignty. Germany and Japan knew they had to control their own supplies to have a military-industrial system to effectively wage war. As Erwin Rommel, the famed commander of the Afrika Korps noted, “Neither guns nor ammunition are of much use in modern warfare unless there is sufficient petrol to haul them around… a shortage of petrol is enough to make one weep.”
U.S. dominance of the oil industry fueled its manufacturing sector and enabled it to produce half of the world’s industrial output during that conflict, including over 650,000 tanks and planes. It built 107 aircraft carriers during the war, averaging two new carriers per month.
This dominance began to crack in 1970, when the U.S. became a net importer of energy as domestic oil production peaked and entered what many thought would be a permanent decline. By 1973, oil imports represented about 30% of U.S. consumption, which would increase to nearly 50% four years later.
This newfound dependency was not lost on OPEC, whose Arab nation state members had long hoped to use what they called the “oil weapon” to force the U.S. to settle the conflict between Israel and the Arab Palestinians on terms they would dictate. In the aftermath of Arab defeat in the Yom Kippur War of 1973, the OPEC nations finally decided to unleash this weapon on the United States, the United Kingdom, and (West) Germany. They could confidently do so because this was the year that Saudi Arabia replaced the U.S. as the world’s oil producer of last resort.
This meant that even at 100% production capacity, the U.S. would be powerless to replace Saudi Arabian supply if it was cut off.
The OPEC embargo caused the price of a barrel of oil to jump from $2.90 before the embargo to $11.65 by January 1974. Americans began hoarding gasoline and were forced to wait in long lines at gas stations. Even after OPEC lifted its embargo, higher oil prices persisted and played a major role in the U.S. suffering from stagflation, the brutal combination of stagnant economic growth and high inflation.
Suddenly, the economics of fracking began to look more attractive and fracking techniques started being attempted in oil basins across the U.S.
President Gerald Ford proclaimed in his 1975 State of the Union address that advancements in the ability to tap shale oil through fracking was a key pillar of his energy strategy while Herman Kahn, the scenario planner and nuclear strategist who inspired Stanley Kubrick’s Dr. Strangelove character predicted fracking would result in the U.S. regaining its crown as the world’s energy superpower.
In 1976, U.S. Department of Energy (DOE) engineers developed a novel technique for drilling for natural gas in underground layers of shale. Instead of drilling vertically, it would drill horizontally. The DOE funded numerous experimental horizontal gas wells, but it was not widely accepted. It also instituted a program that provided up to $1 million in funding for fracking projects, effectively splitting the cost of drilling a well.
It was as a participant in this program that Mitchell Energy attempted its first fracking project in 1981. The leader of this company was George Mitchell, a quirky Texas energy baron who was convinced by a meeting years earlier with the futurist Buckminster Fuller that alternative energy sources would have to be developed or the world would face inevitable catastrophe.
Mitchell had many side interests, including buying 15,000 acres of land near Houston where he hoped to build a utopian city that would eventually be named the Woodlands. He spent his off time playing with an elaborate electric train set, and designing forests for his future city. But his biggest obsession was fracking, which he persisted in exploring despite fierce resistance from employees, family members, and investors. Mitchell narrowly avoided bankruptcy multiple times and was forced to buy out nearly all of his outside investors who were unsupportive of his vision.
Most of the energy industry didn’t share Mitchell’s vision, and the consensus was that using fracking to drill through shale was the “fool’s gold of the energy business.” The DOE had determined a massive amount of shale gas was held in Texas as well as parts of the Midwest, and Appalachia but almost no one thought it could be profitably extracted.
Mitchell’s company was focused on the Barnett Shale formation in north Texas, where the company controlled 400,000 acres of land. Throughout the 1980s Mitchell’s company tried and largely failed to extract natural gas from the formation. One method, which they dubbed “massive fracks”, relied on a DOE-developed technique of pumping large amounts of sand and fluid down boreholes to crack the shale. Other methods included using foams and a “gelled water” approach where millions of gallons of fluid and sand were used. Ultimately, Mitchell settled on using much cheaper injections of water, sand, and a chemical mixture.
It wasn’t until 1997, after 16 years of failed experiments, that one of Mitchell’s shale gas wells proved fracking could be financially viable. But two more key innovations were needed before fracking was ready to take off. The first was a technique developed by engineers at EOG resources, formerly the Enron Oil & Gas Company, of segmenting sections of wells and fracking them in stages using a concentrated shot of fracking fluid to get oil flowing. This was an expensive process, costing roughly $8 million per well versus $4 million using traditional methods, but drillers were able to extract large amounts of oil from dense rock formations that had previously been difficult to frack.
The second innovation was a horizontal drilling technique that had been perfected since its initial invention at DOE. Oil companies could now drill horizontally for two miles without losing track of the shale layer. By this time Devon Energy, which had acquired Mitchell Energy, had developed a near perfect fluid mixture that could frack even cement-like rock. By joining together these innovations Devon Energy saw its oil and gas production levels reach previously unimaginable levels. Nearly 60 years after the first hydraulically fractured well was drilled in Oklahoma, the fracking revolution was finally ready to begin.
Still, there was one missing piece, but it arrived just in time. The high cost of fracking meant oil and gas prices needed to be very high to make it economically viable. By 2005, oil production in the U.S. had reached its nadir at 1.89 million barrels per day and the U.S. was importing 60% of its oil. By 2006, the price of oil had doubled over an 18 month period to $70 a barrel and by mid-2008 doubled again to nearly $150 a barrel. Many industry analysts were projecting $200 a barrel by 2010. The U.S. Energy Information Agency stated that global “peak oil” production would be reached as soon as 2037 after which the world would enter a permanent oil production decline with the U.S. importing 75% or more of its oil at ever increasing costs forever.
The price signal that the fracking industry needed had arrived. A speculative bubble began with Wall Street enabling U.S. oil and gas companies to go on a debt-fueled spending binge, resulting in $280 billion in negative cash flows between 2007-2017 per a Wall Street Journal analysis.
This excess spending resulted in the U.S. producing 7.5 million barrels per day by 2012, an all-time high. But frackers needed sustained high oil prices for their businesses to be profitable. Knowing this, the OPEC nations increased production causing the price of oil to decline to under $50 a barrel by mid-2014 which put the entire fracking industry on life support. At the time, operators in the Permian basin in Texas needed oil prices of $70 a barrel to make a profit.
Facing an existential threat, the fracking industry consolidated and looked for ways to innovate. Engineers began experimenting with a drilling technique in which, after a mile of horizontal drilling, they’d drill sideways for a mile and then curve back to the starting point. This technique, dubbed the “horse shoe”, enabled drillers to pump oil with half the number of wells, essentially cutting their costs in half.
Other new techniques have since emerged or improved. Oil companies can now drill horizontal wells for up to four miles, up from three miles just a few years ago. These companies have figured out that by releasing water at half the normal rate, one crew can simultaneously frack two wells, and with much lower energy costs. Artificial intelligence-enabled software and electronic sensor technology now allows drill bits to be controlled remotely and stay on target 95% of the time.
The cumulative result of this innovation is oil and gas industry productivity has nearly tripled from 2012 through 2022 — the highest gain of any sector in the U.S. The U.S. now produces 20% of the world’s oil and 25% of the world’s gas. This is set to increase next year with U.S. oil production expected to grow another 600,000 barrels per day, pushing 14 million barrels per day of production. U.S. oil companies are now pursuing projects that can be profitable at $30 a barrel, with ExxonMobil’s CEO predicting technology advances could allow it to double its current shale output in the future.
The increased output of American energy producers has been alarming to OPEC, which has repeatedly tried to use coordinated production cuts to boost oil prices. Saudi Arabia is believed to have a break-even price of oil that is close to $100 a barrel due to its lavish spending; that nation needs prices to be higher. But OPEC’s efforts have been fruitless and merely resulted in American producers pumping more oil and gas and gaining market share. Thanks to American innovation, foreign nations can no longer dictate the price of oil and gas so easily without causing harm to themselves.
While most people understand fracking has increased America’s energy security, its broader impact is not appreciated. The Economist has noted the shale-oil revolution has driven as much as 10% of U.S. economic growth since the early 2000s. The U.S. oil and gas industry generates around $1.8 trillion in GDP, more than the entire economy of Australia, the 13th largest economy in the world.
“The cumulative result of this innovation is oil and gas industry productivity has nearly tripled from 2012 through 2022 — the highest gain of any sector in the U.S. The U.S. now produces 20% of the world’s oil and 25% of the world’s gas”
Fracking has also enabled the U.S. to extend energy security to its allies. Following the invasion of Ukraine, the U.S. provided 50% of Europe’s liquified natural gas (LNG) imports, preventing the worst-case scenarios where the continent would be unable to heat itself without Russian gas during the winter. Without such U.S. supply, Europe may have already been forced to capitulate to Russian demands. The three biggest markets for U.S. LNG exports are China, Japan, and South Korea. The reliance on U.S. gas exports in parts of East Asia is a factor in strengthening U.S. alliances in the geographic center of superpower competition.
Because of fracking, the U.S. has been able to largely substitute coal for natural gas. Per the U.S. Energy Information Agency, U.S. natural gas for power generation increased 100% between 2005 and 2012 while coal use declined 55%. This switch resulted in an estimated reduction of 532 million metric tons in CO2 emissions over the same period, mitigating the equivalent of more than 10 percent of 2021 US greenhouse gas emissions. This is more than double the mitigation which can be attributed to the increase in renewable energy generation.
The story of fracking represents a critical type of innovation that is often overlooked. It is the story of making incremental innovations over and over again in the face of difficult obstacles for decades. And those who bet big on such difficult, mechanical innovation have been rewarded handsomely with the profits. One only has to look to 1970’s America or 2025 Europe to understand the devastating consequences of the inability to control your own supply of cheap, abundant energy. It is thanks to men like George Mitchell and countless anonymous innovators in the oil industry that the U.S. has avoided this fate.
