Projects
![]()
No projects found
Investors
New
DAO Treasuries
DATs
Lend / Borrow APRs
Chains
Categories
Spaces
Lets do more throughput than #FrankenDancer!
Recorded: Sept. 22, 2024 Duration: 0:23:55Space Recording
Full Transcription
Good evening on this hopefully for you fine Sunday it's been a bit rainy and a thunderstormy in the UK the past few days
which is kind of nice because it's a bit soothing but missing the sun shouldn't be very good.
So wherever you are good evening to this Sunday stream of mine.
I thought I'd do something a little bit different because a bunch of things have happened over the past few days right so we had uh a long soak test of cassandra which went very well did i think it was 25
billion swaps in about 24 hours averaging i think it was about 300 000 swaps a second
I think it was about 300,000 swaps a second, which was pretty crazy.
And that was a good test.
And then, you know, I kind of make these updates and stuff quite frequently.
But, you know, they are a little bit bland, right?
It's just a screenshot of the dashboard or whatever.
And so there's been a bit more skepticism around that test than there usually is.
And I say usually because, you know, over 10 years of skepticism and it's not possible and yada, yada, yada.
Some of which is fair.
So I thought today I'd do something a bit different, right?
I thought I'd actually take you on a bit of a tool around how a test gets set up
show you some of the workings of cassandra in the dev environment and a bunch of other things
just to give a little bit of confidence to those that may be skeptical that okay if this is fake
it's a very elaborate fake and it would be easier to just do what it says it's doing um and then of course
a couple of days ago at solana breakpoint uh a version of fire dancer which is called franken
dancer which is kind of a mix of fire dancer with um i forget the name that eight gave client for solana it's kind of a hybrid um they did a demonstration
at solana breakpoint of i think they peaked at 1.4 million transfers a second so these were
these were transfers right they weren't swaps or anything it's not very impressive um and i think
it was 100 validators or something like that uh and they sent a bunch of transactions in spam, I guess,
just spamming from accounts. And it was great. And since then, there's been quite a bit of chatter
in our community around, well, how does how does what what I'm working on compared to that, right,
because very much focus on swaps, because swaps are a bit more real world right in terms of um
you know smart contract calls there's actual code being executed it may be arbitrary code so you
can't bake it into the protocol and optimize it like crazy like you can with like just a transfer
right like a transfer is a protocol action basically so you can optimize it um to an
insane degree um but obviously i've been focusing on swaps and defy style benchmarking and haven't
really done a great deal with transfers so it's been this question of you know what can cassie do
in terms of transfers um or whatever the new name for cassie is going
to be um so i switched gears a little bit the past few days because i haven't actually tested
myself uh transfer throughput for quite some time so i had a bit of a play around i wanted to keep
it you know the same kind of configuration that would do the swaps so four core machines low
memory eight gig uh though for the
test i'm going to show you here i've bumped that up to 10 gig because all the machines are going to
be spamming and spamming is quite expensive at the kind of throughput we're going to be doing so i'll
give them a little bit more memory but it's still it's still on the 10 gig right compared to the
fire dancer recommended specs which is like 128 gig or something so um the four core i think the
fire dancer specs are is it 16 or 32 i think um ssd drives uh which are on google cloud so they
actually don't get anywhere near the performance that an actual ssd machine gets because you get iops limited and stuff um so it's not really ssd
io performance so low spec nodes what can we do in terms of transfers so i spent a couple of days
messing around and just squeezing as much as i can out of it and i figured let's do a let's do a
a transfers test.
So that's what we're gonna do. But I'm also gonna take you on a little bit of a tour
of what all this is, right?
So see here, I've got the old obligatory dashboard up.
No transactions have gone through it yet.
It's just started.
It's been up six minutes, steam proposals and things,
32 shards, all the usual kind of stuff, right? All these will start
filling once I actually put some load through it. Then I have a Java visual VM. So you can
see the machines or one of the machines anyway. So I've got a bunch of connections already.
I've got a bag of threads
that do various things so these will populate and start to grow as things happen and i'll run you
through what those are i see in the monitor no cpu usage at the moment there's the 10 gig i was
saying so very little memory right there and then uh i've got you might not be able to see that if you're on a small screen
but basically what a command ready to go here which is going to send some transfer spam
um and i'm going to ask it to do uh i think i've got a billion iterations not that we're going to
hang around for that long for it because we're going to do a lot. At a rate of 100,000 per second across 100,000 wallets with a shard factor of 0.9.
And the shard factor really means I want to favor cross shard transactions.
So if you put that to one, then it just uses an absolute ton of CPU because
when it generates a transaction that isn't cross shard, because you do get them occasionally,
if like two wallets are in the same shard, then it'll be a non cross shard transaction, right?
But we want to do cross shard stuff. So I can say if you come across a non non-crossyard transaction just randomly because of the wallets that you've
picked spend a little bit of time to try and make that a cross-shard transaction like add
a different wallet or regenerate and stuff and if you put it to one then it just yeah cost a lot of
cpu and we don't have a lot of cpu because these are only four cores right so so the shard factor
does um and then this script is
going to run through these ip addresses and the ips.txt file and it's going to send this command
to 32 of the machines that are in this network uh and there is 256 in total so it's going to send
32 requests of a hundred thousand000 transactions per second.
And the network is going to do its best to process that load.
And let's see how far we get.
See if we can beat five answers is the title of this stream.
So I'm going to set that off while I take you through some other things.
So I'm going to play around with this so you can't do those
nice keyboard things oh no that's a mistake oh did the leak key Okay, so this is just going to send these commands and it's going to delay each one by 15 seconds.
So it's going to send one out, wait 15 seconds, send another one out.
And then we can kind of monitor the progress on the good old dashboard as things start to move.
What I want to show you is some of this stuff, right?
So on your left- left hand side that's all
the cassie code so even before getting into it that's quite a lot of code to write for bullshit
right so you know sometimes i do take it a bit personally because i've been working on this for a long time um you know many iterations as many of you know cut probably
many millions of lines of code at this point um and yeah to be told it's just fake or numbers
on a dashboard it's kind of hard not to tell a bit personal sometimes but anyway here's
all the all the code um and then one of the things that I read somewhere was that the swaps code that runs could be pre-compiled into native.
Well, this is the two-asset pool swap code that actually gets run.
And the coders among you will see that that's in javascript of all things not roast or
anything else it's in javascript the reason it's in javascript is because um i can essentially
pass this through the growl vm and use that as an execution engine um in the meantime before I get to bolt in the Radix engine on top of all of
this. And that'll be fun over the next few months because then we can have some of the Radix ecosystem
shut up, Telang. I really should turn Telang on the package off when I do a stream because then
we can actually run scripto code on top of this this testnet but this was nice and easy to get
integrated and to you know start to play around with things uh and it does a lot of things that
you would do in scripto you know you've got buckets and you've got vaults and you're taking
from here and putting from there so the way that you develop uh is very similar to the way that
you develop in scripto i mean there's obviously differences but
the kind of fundamentals you're doing much of the same stuff right so it's quite clips
and obviously that's on purpose right uh and you know one of the other one of the other
little contracts that i run just gives out tokens um keeps a distribution map of who's got what
uh this is the token component which is the native
function that we're going to be having fun with you know so we can create we can mint we can
transfer again some kind of stuff vaults tape pockets etc etc uh and this is the polyglot component, which this is subclass from.
No, sorry, that's subclass from native component.
Polyglot component is what parses the JavaScript stuff.
So we have basically a context pool.
Well, VM uses context.
And when we want to execute some JavaScript, we instanced to get Polyglot
component. And if there is a context available in the pool, which there may or may not be,
then it will either pass the code, evaluate the code, which is this line here, do all
the bindings and stuff. So the execution of the swaps is essentially interpreted
you can do some interesting stuff with growl vm so that it the performance is open you can pre-compile
it to kind of native if you want to but i'm not doing that here as you can see because i'm
evaluating as javascript uh if i get lucky and there is a context available that's just become free and
hasn't been garbage collected yet then i can just jump in use that one set the bindings which
essentially resets the whole thing i don't have to evaluate it again but more often than not i'll
be creating lots of new context as well as they get used um then we machine, which is quite quite a beast, basically does all the execution
of either the contract and the polyglots, or the native components, this just manages all
the state machine and the replication, getting the states, the lock modes on the states for all the
cross shard things, you see shard ID shard id i mean you know just jump around this
right like we could dig into this for hours and hours and hours and hours right but we don't have
to do that on a stream because um here we go oh look we're at 870 000 already i forgot that was
going uh the repo is very close to being made public so you'll be able to go and inspect.
I do need to read me.
You'll be able to go and inspect, you know,
all of the code that's in there
and decide for yourself if it's real or bullshit.
I do need to clean this up quite a lot
because, you know, this is a couple of years
of just hacking around initially.
It was initially research, now it's turned into more.
So you can see
that lots of code smile comments it's tagging up all my uh all my my very quickly put together code
to try out concepts and stuff um so yeah you know so all of these spam instances have been fired off
quite nicely and we're up to 1.2 i need some comments here 1.1 million transfers per second
cross shard ratio is nice and high um so two wallets uh you're on average going to be hitting
um two shards obviously you'll get random instances where you're not hitting two shards and if the
shard factor can't change that then then you'll get single, uh,
shard executions, but it's doing cross-shard stuff.
Um, we've done a total of 200 million transfers in 10 minutes.
We started this at what, six minutes.
Uh, we've got a bunch of pending transactions.
Finality is three seconds from the client.
So if you were on like a mobile device
and you press submit then you would experience about three seconds worth to finality but the
consensus process because there's obviously latency between it going from the phone across
the network to a validator then the validator has to figure out who it's really for and then give it
to them and yada yada um but then
in the consensus pipeline it's taking about two and a half seconds uh each shard is doing 45
thousand transfers per second uh and yeah commit time is nice and low don't really want that above
100 milliseconds otherwise things start to get a bit janky um and around time of uh
quarter of a second so there's a new proposal every 250 milliseconds there's four per second
and each proposal there's a bunch of transactions packed in and there's a bunch of transactions
which have execution completion stuff which means those transactions then become finalized. So that 2.3 seconds is when a transaction is fully final. So
I mean, you know, I've got a bunch of Google instances you can see, well, maybe you can't
depend on what kind of screen you're on. Google needs a dark mode. Oh, I need to switch it to a
dark mode. You know, you can see that they're spread everywhere, right? They're geolocated.
Another thing that people always say, it's just in my bedroom or in my kitchen or some shit.
We've got US West, Europe North, Asia, more Europe, everywhere.
So there we go, up to 1.5.
And I believe the demonstration that someone has just sent me a message in T them about the repo no readme this is how i roll
also yes well i don't need to do a readme yet because it's not public and i know what i'm doing
so the readme is fairly low down on the priorities there will be one there and all the gradle bills
and stuff and everything else that you'll need um but right now i i don't need a rebate so i'm good um as i say uh got very distracted now um yeah i think the fire while the franken
dancer demonstration was uh about 1.4 million transfers per second but they were non-voting so they weren't really taking part in any of the
consensus voting stuff which is fine but like this is doing all the consensus it's doing all
the voting it's doing all the communications around um you know gossip and all those things
where the votes are happening signatures being sent around everything's being executed this is
like full fat right this is a full fat demonstration um and we're now up to 1.8 million uh transfers per second if we jump
over here we can see yeah like some stuff's getting pretty heavy at this kind of pace right
like the atom this is a bit of a rally from a long time ago um this basically this is the
transaction processor right
so you've got the transaction process so then you've got the executions there what the atom
processor is doing is receives a transaction then it passes that transaction to make sure that
transaction is a proper form that it's got instructions in it what are the instructions
are there signatures if yes yes, verify the signatures,
that does a bulk of the work. And then that then gets passed to the progress processor, if that's good, and the progress processor essentially builds the proposals. And so any
transactions that have passed this without failing, you know, they've been vetted, and they
deem to be at least of good form, don't know if they're good transactions yet but at least a good form they get passed into the progress processor and the
progress processor then adds that to a proposal um there's some kind of uh cross-shard pre-consensus
that happens as well so that all the all the shards that are involved in a particular transaction
can signal that they're they're ready to accept that transaction
and then it goes into into a proposal and the local consensus agrees on that proposal if the
transaction is cross-shard then it will be in multiple proposals across different shards and
those local processes will agree okay we're going to try and execute this now, right?
So then there's a state provisioning piece, which is the state pool.
That's where it loads all of the input state required.
And then once you have all of the input state that you need, it then gets passed to the execution thread.
And as you can see, these execution threads are hardly doing anything because as it turns out, swaps really aren't very computationally expensive. Whereas when these are
swaps, these execution threads will be pretty maxed out. And so that's why people like to
demonstrate with transfers per second right because the
execution is very low um whereas with swaps and things the execution is much much much heavier
um but then a quirk one of the nice features that we can do with with this network is that we can
sequence things like swaps because if you have a lot of swaps that are hitting
the same pool you can essentially put them into a nice order so that you're only reading and writing
state once for a bunch of transactions but with transfers unless you have all the state of memory
then you have to go to disk many times um essentially for each
transfer and that's generally what um oh look at that we're over 2.1 million transactions per second
um that's quite impressive i haven't been that fast the most i got earlier one was 1.7 so yeah seven. So yeah, wow. Okay, cool. So yeah, so these transfers might be very quick to execute,
but then you have an IO problem, basically. So then you can do some nice tricks with your
IO, such as like futures and multi thread reads and writes and those kind of stuff so you can get your transfer
pace up to 2.1 million transactions per second which is pretty cool um in terms of swaps like
if we were doing this on swaps then the peak would be seen on a 32 shard network would be about 600
um maybe 700 because i've done some optimizations lately as well. Really quick. We can burst to just over a million,
but it can't sustain. It can't sustain that on swaps.
Sweet spots around 700,000. You can sustain that pretty well.
And this, as you can see, is ticking along nicely.
Now we're 2.2 million transactions per second.
All the spam initiations have finished. How are we doing
on the CPU? Yeah, this is like pretty maxed out. A lot of this is because it's spamming
as well. When it hits the inevitable, this is a single shard execution and then it regenerates it a bunch
of times um it has to rehash it resign it and a bunch of other stuff that's quite expensive
um memory uses thrashy as well but you know what you expect right each of these uh each of these nodes are doing 75,000 transfers per second on a Java VM with 10 gig of RAM and four CPU cores.
So there we go. And we've just ticked over the nice 1.1 billion as well.
So, yeah, I mean, I hope that helps a little bit.
And I hope that can dispel a little bit of the um
of the skepticism i mean you know we could spend hours and hours and hours going through the code
and you know doing all this stuff but who's going to watch it anyway right who's going to watch
dissecting the cassie code for three hours to prove that it's not bullshit when
literally in a couple of weeks after i'm back from barcelona which uh if anybody wants to come and say hey barcelona feel free um the
repo is going to be up anyway and people can dig into it do the builds themselves spin
up their own test networks if they've got if they've got the horsepower to do so um but
yeah so we have achieved our we've surpassed what I even thought we could do. So we've absolutely beaten Solana's Frank and Dancer throughput on real commodity
hardware on a Java VM with 10 gig of RAM, four CPU cores and severely limited I.O.
subsystem because I'm using cheap ass drives on Google Cloud
because spending way too much on Google Cloud.
So yeah, that's pretty much just what I wanted to show really.
I'm gonna go get ready for Barcelona
because I'm flying out in the morning,
got to pack, got to do a bunch of things,
got to have a shave and all these other things
because I've been focusing on this now for a couple of days um but yeah uh
have a great rest of your sunday evening share this everywhere and adios
