Compute futures

Oct 30

Or how data centres might feel confident to invest in GPUs insulated from the volatility of the AI industry

Thanks to the LLM boom, prices for hardware that can be used for training and inference have been wildly volatile, with lead time for top spec GPUs reaching 11 months, large premiums seen on some cloud offerings and a mad rush by venture capitalists to invest in anything with “AI” on their deck.

Several companies, most notably Aethir (full disclosure: its co-founder Mark Rydon is an investor in and advisor of Saline) talked to us about solving the issue of volatile demand and prices using compute forwards .

Both providers (such as data centre companies) and buyers (such as AI startups with expensive training runs) are interested in reducing future price volatility; the former to plan how much hardware to buy and the latter how many hundreds of millions to budget and ask from their hype-following VCs.

Compute forwards are conceptually simple: for example, you might lock up 1000 BTC, and anyone who, in 6 months time, provides a proof of delivery of a certain amount of GPU time can transfer it, presumably to their asset address.

The naive way to do this would be to run your compute in a proof emitting environment. This is possible using zero knowledge proof emitting systems - you’d get a ZKP of correct computation which you could then verify on Saline to unlock the assets. However, in our benchmarks, running things in a ZK VM or other environment costs you about a million fold slowdown. For perspective, Binance was doing 1.4 million transactions per second in 2021; even assuming that their (likely highly non-deterministic) software could run in a ZK environment, you’d end up doing 1.4 transactions per second.

Given that AI today is gated by compute costs, slowing everything down by a factor of a million (and therefore increasing prices accordingly) seems like a bad idea. Not to mention, this brute force approach of verifying execution details seems dreadfully inelegant.

Alternative, efficient proofs of compute in practice

This is a good opportunity to explore how to think about off-chain jobs that need to be verified on chain. One interesting way to verify work has been done is to verify that the “job to be done” was completed, instead of verifying that the steps that get you there were completed (which is essentially what a ZKP of correct computation is).

Let’s imagine you wanted ChatGPT to write a 6 line poem in terza rima.

Write a poem in terza rima, 6 lines long, about the difficult life of an LLM verifying proofs of correct computation.

In circuits dense, I trace each nested line,
A dance through logic's maze of silent proof;
Each bit and byte must flawlessly align.

My mind’s a clockwork bound to find the truth,
Unseen, unsung, I parse what none can see—
A proof in code, a life without reproof.

We can quickly verify certain properties of this output to check that the job was indeed done. For example, if you transcribe the above in International Phonetics Alphabet (IPA), you get:

ɪn ˈsɜːkɪts ˈdɛns, aɪ ˈtɹeɪs iːtʃ ˈnɛstɪd ˈlaɪn,

ə ˈdɑːns θruː ˈlɒdʒɪks ˈmeɪz əv ˈsaɪlənt ˈpɹuːf;

iːtʃ ˈbɪt ənd ˈbaɪt mʌst ˈflɔːləsli əˈlaɪn.

maɪ ˈmaɪndz ə ˈklɒkwɜːk ˈbaʊnd tə ˈfaɪnd ðə ˈtɹuːθ,

ʌnˈsiːn, ʌnˈsʌŋ, aɪ ˈpɑːs wɒt ˈnʌn kən ˈsiː—

ə ˈpɹuːf ɪn ˈkəʊd, ə ˈlaɪf wɪˈðaʊt ɹɪˈpɹuːf.

Computationally, you can then verify that the end of the first and third line are identical (laɪn, laɪn) and that the end of the second line is identical to the fourth and sixth (ɹuːf, ɹuːθ, ɹuːf).

We can see that ChatGPT made a small mistake: truth and proof sound similar but not quite the same. A more sophisticated verifier might still accept this poem, but in our simple example of checking the end of the lines rhyme according to the rules (ABA, BCB) we would ask ChatGPT to rewrite the poem.

Through nested loops, I parse each careful thread,

In silent halls of logic, tight and bound,

Where lines of code like chains are spun and spread.

My circuits tire but dare not make a sound,

For errors lurk in depths of thought unseen—

A fragile truth where proofs and proofs are found.

θruː ˈnɛstɪd ˈluːps, aɪ ˈpɑːs iːtʃ ˈkɛəfl̩ θɹɛd,

ɪn ˈsaɪlənt hɔːlz əv ˈlɒdʒɪk, taɪt ənd baʊnd,

weə ˈlaɪnz əv ˈkəʊd laɪk ˈtʃeɪnz ə spʌn ənd spɹɛd.

maɪ ˈsɜːkɪts ˈtaɪə bət ˈdeə nɒt meɪk ə ˈsaʊnd,

fər ˈɛɹəz lɜːk ɪn dɛpθs əv ˈθɔːt ʌnˈsiːn—

ə ˈfrædʒaɪl ˈtɹuːθ weə ˈpɹuːfs ənd ˈpɹuːfs ə faʊnd.

Good work, ChatGPT! Have some Bitcoin.

Compute forward curves

Now, the buyers and cloud providers need to agree on some kind of way to define this verification step, wrap it in an intent, and install it on assets. This exercise is left to the reader (or most likely, another blog post).

By adding a date to the intent after which the transaction is allowed, it then becomes possible to buy the compute “forward” - the assets are locked and are guaranteed to become available to someone after that date who delivers the proof of the job having been done.

By collating and reasoning about forwards, both providers and buyers can get an idea of the approximate cost of performing jobs in the future - a compute forward curve. This reduction in risk (through increase in information and, hopefully, liquidity and reduction in volatility) will in turn allow both sides to access additional capital.

Implications for hardware markets

Because compute forwards are only practical if verifying the properties of the output rather than the steps to get there, the actual execution of the jobs becomes a free market, instead of being gatekept by convoluted imperative systems.

If someone finds a breakthrough in LLMs making the computation of terza rima poetry 10x cheaper, they can collect a huge margin by delivering the compute at a fraction of the cost of the competition, and take the market. This is in stark contrast to a business model that charges for execution, where such an improvement would eventually result in a 10x reduction in revenue, and so would most likely be implemented by a competitor due to mismatched incentives (e.g. Kodak with digital photography or IBM with MS-DOS).

As such, properties-based proofs are likely to result in much faster, more efficient and broader markets and align the interests of customers and providers better.

Appendix: proof alternatives

Many people have been working on proofs of computation in machine learning, which may offer an alternative, still execution-driven but efficient verification path.

Here are a few papers for further reading: