Therein lies the problem. "Too big to fail" is "too big to allow" from a risk management perspective.
To me, it seems that the financial system has gradually changed over the past 40 or so years from being a means of facilitating the "real" economy into a parallel world in its' own right, failure of which seems reasonably possible and involves catastrophic consequences for the "real" economy in addition to the financial world.
As an example, over 1 billion barrels of oil are traded "on paper" every day and yet the world produces only about 90 million actual, real barrels of oil that you can see and touch. That level of trading goes far beyond anything reasonably necessary to enable the buying and selling of real, physical oil and has primarily become an exercise in money shuffling. Go back to 1970 and most oil was sold under long term contract - that's one trade not per barrel but for the entire oil field. And suffice to say that there was plenty of oil being used in 1970, the lack of money shuffling didn't stop the geologists, drilling rigs, pipelines, ships, refineries etc from working.
Yep, agreed on both fronts.
So the powers that be are trying to ensure that no node is important enough to take out the system. Furthermore, they are working to ensure that no group of interlinked nodes can take out the system. They are also trying to stabilize it by ensuring that banks have more regulatory capital with Basel III and all its variants. Principal risk has become so expensive in terms of capital requirements that the major investment banks have largely shut these down, becoming brokers in a purer sense. Risk taking activity has moved more towards funds management and hedge fund management. Perhaps they'll be in a better position to take it and are more transparent in terms of credit provision for leverage. There are also moves to centralize OTC derivatives and have them cleared and collateralized. If achievable, all of these will reduce systemic risk.
What you describe in terms of derivative activity relative to actual underlying activity is crazy isn't it. But it is symptomatic to sharing the burden of risk across several counterparties and increasing interlinkage and 'complexity'. It's partly for the same reason that FFX activity vastly outstrips world trade. However, it is also symptomatic of excess speculation.
Today deals like the Chinese purchase of Russian oil are also multi-decade arrangements which are akin to your oil example. However, the volume is contracted but price fluctuations probably appear somewhere. Same deal with Japanese long term contracts for our LNG. Some of this will still need to be hedged by gas fired generators in Japan or bulk consumers of electricity. Risk sharing, liquidity and securitization are important. Whilst things can happen without it, much more happens with it....except some of it is bad. How do you sort out the wheat from the chaff? It's pretty tough when the system seems to want to find a way to turn into chaff when there is a buck to make. Still, you gotta try and restrain it. However, dragging it back twenty or forty years would very probably squash our standard of living quite a bit more than the GFC did.
Interestingly, part of the problem is that finance is becoming very 'complex'. Everything is linked to everything. No-one knows how the whole thing works. Prod here, something, somewhere pokes out. But you don't know where.
This is happening to the world production economy. Production chains are getting longer and longer as various productivity enhancements are sought around the world. As a result there are an ever increasing number of parts generated all over the place to produce a single product. When one link goes, the whole production line stops. This happened to car production around the world for certain makes when the Japanese tsumani occurred. It was an indication of the vulnerability of our production chains. Similarly oil is too big to fail.
It's all a very delicate dance and it is unstable (on the border of chaos and complexity) in nature.