Operational Risk and Where To Find It
Operational Risk and Where To Find It
The type of risk everyone thinks about last is the one that costs the most
In addition to founding a project to fork OmniLayer for Litecoin and adapt a derivatives Dex on it, I also have managed money for a long time. I have seen 6-sigma events happen on a frequency that makes them 2.5 sigma events, and I have seen the mighty humbled. I have seen people whose Twitter handle dared on to stop them, be stopped, by leverage, and I have seen the biggest fund in the space rendered a 1.5B USD crater in many balance sheets, I have tempted fate once or twice myself. I have incurred losses, but most of the time, these losses have been due to operational risk.
Sadjyit Das in his classic, Traders, Guns and Money, sums up 4 kinds of financial risk:
price risk - that ol’ classic, price go down, lose money
liquidity risk - we bought a small-cap stock in size and can’t exit with simple order flow, a sophisticated executing algorithm would take time to distribute
basis risk - we put on a “delta nuetral” position of long futures, short perps, and now we are paying negative funding and underwater on the premium, it even went against us into backwardation!
operational risk - I have discovered a hard drive containing 300,000 BTC and will submit it to the authorities
Reading Das’ satirical memoire of his own career prepared me, sitting in an office in Buenos Aires prototyping potential NDS games and clicking on Oanda, to one day have a great career as an indie crypto financier. 7 years of money management later, with a few seasons of successful speculation in between, I can say that it pays to be skeptical. And there is nothing more righteous to be skeptical of than your own mind and dilligence. Always double check! Always think your thinking through.
This is why Ray Dalio is such a philosopher, he has conceptualized operational risk at a deep level, into his own cognition.
A wise man knows that the market is Truth and can do *anything* at any time. A wise woman managers her counterparty exposure and discerns their credit risk frugally, withdrawing first to withdraw best when trouble emerges. A wise numberphile understands that charts are narrative summaries of a series of quantum events involving orderbooks, latency, fiber optic cable and microstructure.
Before I make myself sound too wise, here is a quick summary of my major operational mistakes:
Mistook the margin levels, trying to limit lock-in to CryptoFacilities due to a limited-flow AML policy and my name not being on the electric bill (‘twas a holding company), got liquidated first thing in the morning and scrambled to rebalance, lost ~8k
Juggled multiple accounts and scrambled into longs on both when I got a slight time-lead on the news drop about Segwit 2x being canceled. I neglected to manager the smaller account and took a loss of ~6k.
Did not flag reduce-only on a limit order, leading to a lossy position, -3k~
Did not have available on an exchange a reduce-only, *and* forgot to cancel, the resulting long position earned about +100k in about 30 hours for the fund and carry for me.
Someone was leveraged and did not think about the risk that the exchange would have a temporary outage of market maker bids, leading to the system marking price in a flash crash lower, triggering liquidations into my orders, losing to the fund, +21k.
Did math wrong and was over-hedged/long relative to benchmark, -16k; there’s only so many maths to do in these rotations so that’s a one time thing.
Accidentally clicked multiple times thinking the UI was unresponsive and pummeled a thin ask book with my market orders, instantly losing ~3k
Did not appreciate the importance of toggling fixed vs. cross margin on Mex, trying to make a spread, insta-liq -$600
Another exchange had a laggy front-end and appeared to cancel orders that were not, leading to two -3k tarde trades; -6k
version control deployed the wrong script, causing a market maker on an alt coin to crash the price, -3k
Martingaling at any time is based on farcical belief, ignorance, or weak will, and is thus always and everywhere an operational failure of the mind; -5k, -2k, -10k
Bug in code looping too hard, well, that’s going to cost a few hundred bucks if size is small.
All told, I have made more money from operational risk than I have lost, in my career. This is due to a small miracle and is a pillar of my personal faith, but most hedge fund managers can’t rely on that!
A Hedge Fund is meant to fund the hedges, so that you capture the basis differentials to extract uncorrelated alpha. In the stock market this meant doing analysis in the 50s or computer models since the 80s to mass-trade long/short different stocks, and make money on their differentials. That is reasonably hard to do successfully. In cryptocurrency this can be done by either capturing a pair of trades that have some after-fee profit margin, e.g. classic arbitrage, or by holding instruments that pay a difference in funding rates - interest rate arbitrage (which is only 70% arbitrage at best) - and also by holding instruments long/short and trading the range in their relative prices (not even arbitrage, risky! But smoothed volatility).
I like hedge funds, and I like that anyone can start one these days and go pick at the ashes of crypto spreads now that giants have fallen and created the new continents with their carcasses. You don’t have to get into a hierarchy to make money in finance. If you are very skilled as a coder plus you have the numerate insight into the markets that makes a great obsessive quant, you should do as 3AC in the early days, run mostly your own capital. If you borrow, for the love of We Live In A Society, put the money into something with similar duration, low credit risk, and now price risk.
Well if you are avoiding credit risk, duration and delta, how do you get any yield?
What I like to do is find safe stuff and then take advantage of plentiful leverage to scale. Then there is the problem of moving collateral around. Automating withdrawals or DeFi movements is the height of operational risk, because now you’re interfacing with private key signatures or 2FA deferred API calls. Automated trading on Dexes is problematic, but *not* unsolvable, here are two simple solutions for Litecoin/Bitcoin using TradeLayer:
1) Put your funds on a multisig, think of it like a subaccount, and use multiple servers holding the keys. Pre-sign unpublished retrival transactions that sweep the token balances out, refresh when older inputs are spent, with new inputs.
2) Do the above but automate javascript execution of RPC calls to hosted nodes, using Smartweave contracts on Arweave. It’s like "web5” in the sense that it’s combining web3 (the BTC tx signature and the smart contract soliciting it) with web2 (hosting back-ups and defending their attack surfaces, keeping uptime).
3) Go wholesale by bridging tokens from TradeLayer to another environment and doing full automated market making in a smart contract.
Much of the industry went straight to 3, and build architecture to make that path. But, what if we thought about the mechanics of decentralized exchange in a more basic way? In order to create an on-chain TradeLayer transaction, I just need a private key, a UTXO input to cover the fee and put it on-chain, and some TradeLayer asset to use. In order to L2 TradeLayer, you make a multisig with someone, commit tokens to it, and start signing trades. The 1 and 2 solutions above do different degrees of dilligence to limiting attack surface through automation and attestation. 3, in the form of ZK attestations, creates a composability to the rest of the EVM-based chains. It might make operational sense to have more composable, atomic elements of value and control that can stack up to the fully automated solution, rather than use a singular virtual machine interpretation.
How many billions have been lost to DeFi hacks? Many of these were based on code errors. Some were based on logical/math loopholes in the system dynamics. A minority were based on exploiting interactions with deployments and contracts. The way our components interleave can be optimized for developer ease as in traditional web2 development, or they can be optimized for operational failsafe resiliency.
Based on recent events, the sobering realities about peg economics have been made clear. Luna had a negatively convex, death spiraling financing-like mechanism, where it would double in supply for every 1/2 the price fell, under some threshold. The USD/UST peg breaking exarcerbated (raised) that threshold as the price fell into it, and the games began. Even if the operational risk of the pool deployment were just utilized as a momentary weak spot where the lower UST prices could be printed, once they were and there was not enough bid to buy back all the liabilities of the system, we entered the death spiral. Price risk became convex, and operationally they were not prepared, other than to cease trading on Binance (centralized) or close the redemption window (also centralized, admin keys are a crutch). The incredible speed by which this arbitrage invitation could be executed lead to hyperinflation in a short time. *That* part was inevitable. However the impetus of the descent, a bad print in size triggering a cascade, that’s bleeding price risk and basis risk into an operational risk for the entire user base of a platform.
The analogy for TradeLayer here is that ALL and LTC trade and create a VWAP for settling, each block, the perpetual swap on ALL/LTC. ALL + - ALL/LTC => sLTC, people mint these, they can use them as collateral for oracle derivatives or other native derivatives. What if someone wanted to aggressively wash trade? They get a number of “votes” each block based on their position size of what to weight the VWAP with, and pay 1 bps, that’s too cheap.
Contemplating the reflexivity of the UST system, we went to work making a function that filters on the VWAP used for mark price. Based on trailing measurements of volatility, we increase a parameter N, which starts at 1, and we eliminate from the sample all trades from addresses that have traded within N blocks. In this manner it will become impossible to spam the system with wash trades. It would also filter out some rapidly turned over volumes. N increasing also shifts the weighting of how much of the mark price comes from channel trades, and how much comes from on-chain orderbook trades. The on-chain orderbook is from Mastercoin 2014, updated somewhat, it has higher fees, a big maker rebate, and a slow confirmation with mempool uncertainty for timing. However for longer-term standing orders, it’s a great way to market make knowing you can predict when you’ll get a confirmed match, and hedge that flow for a good net-profit including the rebate. Thus, the on-chain orderbook takes advantage of the Sybil protection of the blockchain itself, which is bad for general trading but good for a secondary layer of base liquidity. We plan to add an AMM-like logic to the insurance fund where it will hedge 50% of its ALL supply and put many orders into this on-chain book to maintain that 50%, in addition to buying liquidations that go unfilled.
Additionally we’re doing a limit of how much the mark price can move on a given block’s settlement.
However with all these systems, an inverse-quoted mechanic is going to liquidate all open interest assuming it’s 1:1 at a 50% drop, a bit higher in fact, due to insurance fund fees that act like insurance premiums. If the liquidations go badly, the insurance fund loses money, if they go well and are filled, the premium is taken and slowly grows the fund. In a sense, an insurance fund is a de facto AMM/hedge fund that does only micro-trading arbitrage, and how successfully it does so, it mostly a matter of liquidity risk.
Jump provided all the liquidity you could want to Luna, but one pinprick of operational risk exploited by aggressive shorters lead to a risk of ruin. If Luna had better math, not an exponential issuance, it would not have been so fragile. ALL and TOTAL have limited supply and any amounts that get squeezed out of the insurance funds due to a crash, will at most, increase the float by 30 or 50%. Maybe in the future if 90% of the coins are bought up into the insurance fund by high volumes on oracle contracts generating fees, then the float could increase by 10x, and that would be sad to see, but survivable.
Ultimately we’re building a protocol to enable anyone to do the same business that many others are doing, derivatives exchange, and that makes money. By factoring out costs and decentralizing it to basic components, we get fees down lower than the rest of the decentralized exchanges or centralized exchanges as of 2022. We hope the industry will get more competitive on costs and that this leads to dramatically greater liquidity and market efficiency.
Having experienced the gamut of operational risks that can befall someone in this enterprise of running positions, I wanted to build very good atomic tools to compose robust algorithmic trading systems. That’s what we’re launching.
There aren’t a lot of articles on operational risk but here is some further reading.
This is a similar article by someone who manages bigger size.