Projects
![]()
No projects found
Investors
New
DAO Treasuries
DATs
Lend / Borrow APRs
Chains
Categories
Spaces
OK enough is enough! Time to moon the bar & end the scaling BS!
Recorded: Nov. 13, 2024 Duration: 0:34:31Space Recording
Full Transcription
everyone i'm back at home after lots of travel over the past few months and i've been getting
quite deep into the all the cassandra stuff because i know everybody is very eager to see
what cassandra could do in the public tests so before i get into what this video is about
what stream not a video it's live uh so recently now i'm back on home turf for a
while i'm getting back into cassandra and the testing the main thing i've been doing over the
past week or so is uh cleaning up code making sure it's all nice and tidy so it can be open sourced before um the test happens
and doing this thoroughly you find things right that you can improve or little tweaks you can
make little optimizations and stuff so it's not explicitly about performance it's just about
optimizations and stuff so it's not explicitly about performance it's just about good quality
code as much as it can be in the time that i want to get it done it doesn't need to be perfect it
just needs to show that cassandra is doing all the things that you would expect a main net to
be doing so consensus signatures validation, all those wonderful things
that you would expect the network to be doing
so that folks, when they take a look at the code,
they can pick through and, yep,
this is doing what it's supposed to do.
So that's what I've been doing.
But no need to be perfect, Beam, exactly.
Perfect is the enemy of good.
But it needs to be good enough so that people can look at it and say yeah this this is legit stuff so that's most of my workload on
at the moment is getting the code good so it can open source but we find things along the way
um and so i found a few things believe or not, in my terrible prototype research code.
Not a great deal, but stuff that's good enough to change and spend the time on and push things a little bit more. I remember before EBC, which seems like forever ago, I did a test, Ied and i think it peaked at 2.2 million
which was pretty cool but peaks aren't the same as a sustained throughput right so a peak is great
but it can't do that for very long like 10 15 minutes maybe half an hour, but then eventually all the nodes get overloaded.
So at EBC, we had a demo of Cassandra running on the booth,
and that was stable at about 1.2, 1.3 million.
And it did that all day long, right?
So sustained throughput is what really counts,
because that's what your network is going to be doing, right? Sustained throughput. You might get a peak now and then, but sustained throughput is what really counts, because that's what your network is going to be doing, right?
Sustained throughput.
You might get a peak now and then, but sustained throughput is what matters,
which is why I don't think a lot of these tests that are done by other projects,
I'm not going to name any, maybe, don't know.
But they only run for like a few minutes, right?
It's not a real test.
So 1.2, 1.3 million, pretty good.
Pretty good.
But I know we can do more.
I knew we could do more.
And that's what I'm going to show you today.
So all you guys speculating about all the other things that I was going to announce this evening,
unfortunately, those things move slow. But there are some things that can move fast and this is one of them so what we're going to do tonight is we're going to run a test um which i'm going to replicate with
the small community group in a couple of days because that's always what we do just to make
sure that my i say local but these are geo
distributed nodes anyway and they're in different continents and it looks like a main net right and
there's a bunch of validators and they're in different places and they're all fairly low spec
and stuff but i always like to run it significant improvements and stuff past the community group
just to make sure that you know it's legitimate so probably do that in a couple of days um but we'll do this tonight because this is a fun thing to do and i've been quite excited
to show everybody now before we get into it there's a few things i just want to run over right
so as of right now let's bring this up i've got a network i've just started like 10 minutes ago
um that's running a pathetic 500 600 000 transactions per second
still nobody cares about those numbers anymore well they might care about this number especially
when this is replicated in the public test,
and then you guys can all go and shill Sheehan and Cassandra
and the roadmap and all these other things that are coming.
So hold on to your horse and just bear with me.
Doing a lot right now, and this is one of the things I can demo today.
So if you don't like it, log off and go and drink some whiskey or something.
I don't know, whatever.
After that, rude interruption by mr stallion so we got a network running 600 000 right and
this is the same kind of thing as it always has been geo distributed we've got a bunch of nodes
in here in different locations i've got some in different data centers as well.
256 of them.
It's no longer a nice 2.6, two second consensus period.
And then we look here.
And this is a couple of the nodes I've got running. So it's not costing anything.
And these nodes are a little bit more powerful
than the ones i usually quote so usually i say four core eight gig right the reason these are
a bit more powerful is because these nodes also do a lot of the spam the transaction generation
and that's really expensive um so these are eight core and they have a little bit more memory just
so that we can do spam on the validators which isn't ideal and we'll come back to that a little
bit later on um so doing nothing right now 500 000 600 000 transactions a second so they're not swaps i'm not doing swaps right now i'm doing tps
um simple swaps but i think that's fine right now we can do a swaps one later on another day
um and you can see we've got a bunch of threads going there's connections and stuff and it's just literally doing basically nothing right so we've got
they're doing basically nothing right so we've got
600 000 nice and steady easy peasy but we've seen that that's all news so
let's bring up a window and let's go to a different screen and we're gonna run another
so i named this script after you beam because you were saying to send it so we're gonna send it
i've got to put this damn sshp in a million times a day which is always fun so this is now going to
million times a day which is always fun so this is now going to trigger some more spam
on some of the other validators
and we will see an increase in tps so wait there's more to come why am i using windows
because windows is convenient for just stuff and then i'm using
linux for actually running the network don't i'm old school been around a long time windows works
well for me for my day-to-day and all the documents and crap that i have to open for foundation hats
and rdx hats and all these other hats i have to wear right sorry hardcore i know i know you all
use max these days but i'm a windows linux guy so
forgive me so this is going um it's got a bunch of nodes it's going to send some spam through
then if we look back here we'll see in a minute this starts to creep up as the spam gets injected into the network. Now, I'm going to explain this bit while this is ramping up.
So at the moment, what I'm doing is I'm driving the spam from the validators,
which isn't really ideal, right?
Because in a real network, your spam would come from clients so people on phones
and browsers and all those other different things doing their transactions through those
the transactions obviously wouldn't be generated on the validators but to
basically save a bit of time effort and money until this point it's been more convenient to run the spam
on the validators but the issue which you'll see is that the validators are basically saturated
because it costs more to do the spam than it does to do the consensus and all the other stuff that it needs to do for these tests so
the numbers that we're going to see this evening aren't really reflective of the compute power
that's in the network because most of the compute power in the network is actually being used to
generate the transactions to spam the network so i'm at the point where I need some dedicated spam machines instead of running the spam on the validators.
And that will allow a significant uplift in throughput, both in terms of swaps, transfers, and basically everything, right? So if we look on this node right now, see these,
these are the spam threads, right? They're just completely saturated. Look at the CPU,
it's bumped up. Memory use is quite erratic because it's expensive on CPU and memory to do spam.
As the load in the network increases, this will get more erratic,
and the validator will spend most of its time, one, generating the spam transactions,
and two, doing garbage collection on memory that's no longer needed.
So what we'll see here in threads, these will stay completely green in the CPU time.
And these, which are actually in the real work, Atom, Processor, Execution Thread, Progress
Processor which generates the proposals, Event Thread, Event Buzz and stuff, the gossip,
gossiping between nodes and the network, state process, all these things, these all stay quite low, right? So when we're at full chat here in
about 10 minutes, you'll see there's plenty of capacity left on these threads. So if I move
the spam generation away from the validators, then we can then utilize properly the available capacity and these actual threads that are doing work around consensus execution and all that stuff.
That's the next step.
But even with all this going on, even with the validators doing all the spam, which is mega expensive, we're going to see some good numbers really good numbers and move the spam out and we
can we can expect approximately a two to three x um increase in throughput if the spam has moved
away from these validators so that's not going to be something i'm going to prioritize but when i'm
at a convenient point to move spam away that's what i'm going to prioritize but when i'm at a convenient point to move spam away
that's what i'm going to do and then we'll do one of these streams and we'll drop some more bombs um
so this spam is being called in nicely so we've got a bunch of these to go through about halfway through at this point
and this is going to settle about 1.2 1.3 which is where we've already been in the past right and then i've got two very beefy machines and i'm just going to send an absolute bag load more of spam in and we'll see what happens and what number we get so
while this is going on for a minute
any good questions in the chat do a live launch of making the github repo public yes that is a good
idea to do as well beam you're full of good ideas happy birthday by the way uh you must be
about 3 000 years old by now uh slayers came here for a lego update dude i'm not buying any more
lego until i've had the time to build some of that over there okay i've got a big pile there
and there's another pile in the house i'm not touching any more lego right now are we slightly less early now no we're not because well we might be I don't know but
the reason for this I'm just I'm tired of this scaling bullshit that just goes around
everywhere altos are the solution side chains are the solution, sidechains are the solution, monolithic, FidAnswer, all this stuff, right?
None of these things are going to be able to get anywhere even close to the numbers you're going to see
on this somewhat crippled network where the validators are actually producing all the spam.
Nowhere near, nowhere near.
So I would say you're still pretty early because this is just going to keep raising
the bar right every time we do one of these tests from this point forward it's going to keep raising
the bar every time we do one of these tests it's going to give you an idea of what we can do
in the very soon public test and beyond that as well like it's not just going to be one public
test there's going to be multiple of them over the period of like six months while i'm working
on all the radix engine stuff and bolting all that stuff on top of cassandra there's's going to be multiple of them over the period of like six months while i'm working on all the radix engine stuff and bolting all that stuff on top of cassandra
there's just going to be lots of these tests where we just keep pushing this bar we're going
to be in the millions by q1 of next year and hopefully we'll be pushing tens of millions by
q2 and then that just stops the train on the tracks for everybody else i'm just tired of this
the training the tracks for everybody else i'm just tired of this crap around scaling is solved
it's not solved it's nowhere near solved here's a good question am i hal bent on fast tracking
shian now this is amazing if we can get the word out shian is my one two three four five six seven
eight nine ten priority it's just it's all I'm thinking about.
Obviously there's some foundation stuff
I'm figuring out right now with the separation
and everything, but that's all gonna be wrapped up
in a couple of weeks.
So it's gonna be like roadmap, Radix engine integration,
figuring out what needs to be built with Xi'an,
organizing all the logistics around that,
papers, technical documents around Xi'an Cassandra consensus how
it bolts into the radix engine all those things there's an absolute shit ton to do but i'm quite
happy to continue spending 18 20 hours a day sitting here to get us to a point where Xi'an
is not just on the radar it's absolutely inevitable because it's it's time to show everybody else
exactly what we have and being on the sidelines and watching this bullshit every day it's just
yeah it's it's annoying me sorry everybody a bit of a rant there but it's i think it's warranted
um these are these are transfers right now um swaps we can do a swaps one but the swaps one
we did a swaps one a little while ago right where it was um 700k 800k and it had less of an impact on the socials than when we did the stable 1.2, 1.3 million.
So I think still, even though it's less legitimate, I suppose,
actual raw TPS, bigger number, has more of an impact than legitimate use cases like swaps,
which, again, is annoying and it's bullshit, but you've got to play the game.
So we can do a swaps one as well in a few days if this gets adequate attention.
We can push that out too.
I would imagine the swaps will probably well over a million as well at this stage,
like well over a million.
Maybe even chasing down these kind of numbers
i've i've got a bit of work to do on on the swap stuff so that it's comparable again but
it's still an insane number i mean look we're doing 1.3 million 1.4 million just happily
and what's like you can see he's filling up a little bit but nowhere near to the levels where the cpu
is saturated most of the work is still on the spam but you know rewind 10 years or more and
everybody was shooting for like visa scale right and even myself for a little while until i actually realized no you need a
hell of a lot more than that a few thousand transactions per second with capable peaks of
maybe 50 60 000 for short periods of time and then fast forward now and we're just twundling along
happily at one and a half million tps it's actually insane right like even from my perspective
it's insane um to finally be here at the point where there's solid tech that does this stuff
anyways right so about 1.6 million sustained nice and happy which is better than we've had before but in the immortal words of beam earlier on
send it just suddenly started playing crazy okay so let's send in a bit more and set the bar
for a week so everybody can try and catch up.
And then we'll do another one where we send it even more.
So I've got a couple of beefy spam machines I'm just going to spin up now.
I was doing this so I can just test out some of the ideas around moving the the transaction generation away from the validator
nodes but they're not perfect i need to do a bit more work on them but these have just been
hanging around for a while not really being used so we'll use them today to make a statement
uh okay so in a few minutes you should see this thing go parabolic to a particular number and i know
what that number is but you guys can find out as we go uh slays if i remember correctly you were
involving some educational facilities in the open public test assuming that sort of thing still has
to boot in ahead of time do you have a date for the public test and will the open sourcing
coincide with that same date so there is yes a little bit of lead time on getting everybody
arranged which is happening um but i'm not going to keep the horse in the stable for much longer, let's say.
So in the best case, I want to pull the trigger quite quick on this at this point.
This has taken too long.
A lot of things have got in the way.
A lot of distractions over the course of the year.
It's time this was done.
So if I can't get everybody's ducks in a row for the same day, then we will probably do a community test ourselves just to set the stage and then bring those institutions and third parties in on a slightly later date.
on a slightly later date doesn't really invalidate the test because the purpose of the test was
public community everybody can run a node all that stuff all that fun stuff so we can still do that
code will be open source before that happens there's a bunch of technical blog posts that are done as well, waiting, which explains Cassandra on various degrees of technical
depth, so people can at least have an idea of what Cassandra is doing. It's quite difficult
to get the number of parties that are involved aligned quickly, so that's not possible. We'll
run it before, without them, and then we'll bring them in on a subsequent one because there's going to be multiple ones.
Obviously, preference is to get everybody lined up and in before the end of the year.
But yeah, we'll play it as it goes right now.
now um so that's all the additional spam gone in and we're now running along a nice 2.6 million
transactions per second and i'm going to let it just run that while we continue to well while i
continue to talk and you guys continue to post interesting questions like this one how about formatting the
numbers with separators for k and millions yeah it is a bit hard to read unfortunately
I still haven't got around to this it's one of those very low priority things but I will
endeavor to get it done ASAP.
Let the horse out of the stable.
Yes, it is time, Lucas.
Absolutely.
This is ready to go.
So let's go.
Why would people care now?
You've been pushing TX and swaps with Cassie for God knows how long.
Well, there's one little thing that's happening right now that maybe you haven't noticed. Bitcoin is at all time high. The amount of attention on the crypto space is going
to go insane over the next few months. Cassie has always been a scalability shot to solve problems,
issues, things to figure out for Xi'an, which it does.
So why not use it to capture some of that attention that is inevitably on its way?
This is late, yes, but maybe actually the timing is right.
We'll see.
We've been a deep bear on sideways all year as an industry.
If you want to grab attention, now is probably the time to do
it so maybe it was meant to be this way um so 2.7 2.6 million transactions per second now
let's just look here right so you can see that these actual work threads that are doing consensus and stuff
they're just underutilized right so this one is like 22 utilized this one's 20 utilized
cpu has jumped because this has got really thrashy now and most of this
is caused by spamming so if we take theming out, this will drop down to some nice levels
about here, all the way along. And we've got a good two to three x on top, which we should be
able to extract from these validators, which means that even though this number, in my opinion,
is very impressive, because we're sustaining 2.6 million transactions per second
even a 2x uplift on that gets us into the 5 million transactions per second range which is just
absolutely insane and if we can get more than that then we're heading into seven six seven million transactions per second territory still on a network of 32 shower
groups i know you can't see it because my stupid
face is in the way but down here where my head is which i don't think i can move on restream and
maybe no wrong way okay and now i've completely you can see it now right um
now i've messed everything up you get the idea we've got 32 shard groups um you double that and it doubles again so if we can extract
why can't i get my face back up if we can get a 2x uplift on this then we're punching 10 million transactions per
second with a shard group count of 64 which is interesting because how do i switch this damn thing
around i don't know what the hell i'm doing here
don't know what the hell i'm doing here anyway
it's been a long day it's it's a long it's it's 11 pm i've been up too many hours anyway
10 million tps is interesting because that's exactly what multiverse experts do that they
could do really around today and while i like multiverse x very much and I like Robert and all the guys they haven't
done that nobody's done that nobody's done 10 million TPS so bump up to 64 shard groups when
the time is right move all the transactions spamming away from the validators onto dedicated
nodes so it looks more like clients and we're actually in a shock with hitting 10 million tps over the next few weeks so on that note i'm just going to leave this going a bit more and i'm going to look at
some more questions because there seems to be quite a few
i fear that if we don't gain traction in this cycle, the project won't be able to sustain itself until the next cycle, even though, even if we don't go wrong way, we start to get a picture.
Luke, this ain't going anywhere, right?
Let's just say that we don't gain traction.
Traction in the sense of, I know what you're thinking.
Moon. Whatever that means means I'm not sure like there will be traction to post the Cassandra testing post a bunch of other things
that are coming down the pipe the proof of concepts that are being worked on Anthic blend
all those other things.
There will be traction. Will that traction be as much as you want it to be? Maybe it won't,
right? And that's fine. But we've been around. We're not going anywhere. I'm not going anywhere.
No one's going anywhere. This is going to get done one way or another. This is not the time to
quit. I mean, you guys can if you want to. I mean, that's your
prerogative, right? But I ain't going anywhere. This is going to get done. I set out with a mission,
a task, and a challenge, and that mission, task, and challenge is going to get completed one way
or another. So yeah, maybe we do miss this one. Maybe we don't. It doesn't make any difference to me.
Plenty of one way.
I'll still be here.
Dan, open source CASA tonight.
No, not quite.
There's still a bunch of stuff I want to do to tidy it up.
There's a few things that aren't clear in the code,
especially in the critical sections. So I've got a nice long list of things that i need to just make a bit more human readable so if
anybody does go digging around in there and they want to check on all the fundamental things and
that they're doing what it's supposed to do they can find them and they can understand them so
there's still a bit more to do but not a great deal uh what's the upside of open sourcing cas at this point well so people can
verify right people with scales can go into that code base and they can verify them audit companies
can go in and verify them third parties that might be interested in building on radix and xi
and forms part of their rollout strategy in the future they can go and verify that we're not
full of shit basically right and that this is doing what it says it's supposed to be doing
is it going to be a code base you could spin up a main net tomorrow no of course not because
there are certain things where corners have been cut for efficiency because it was just a
quick and dirty research network at one point
and it's turned into something much larger that's fine right are there are there security holes in
there yeah sure but it's not meant to be secure it's meant to show fundamentals it's meant to show
it's doing everything it should be doing it's meant to show that serverless works it's meant
to show that the cassandra consensus works it's meant to show that the Cassandra consensus works. It's meant to show that you can achieve huge scale on low powered nodes in a decentralized
and distributed network. That's what it's meant for. If you want a production grade
mainnet based on those ideas and that research, then that's what Xi'an is. Xi'an will be secure.
Xi'an will scale. Xi'an will be all the things that it needs to be
for a main net cassandra is a research test net so it shows what it needs to show to prove
viability and that helps grab the guitar and play something you know i wish i could but i haven't
played it for so long i don't have enough stretch in my hands and fingers, and the tits of my fingers are as soft as newborn skin. So can't quite get my cords. And I keep picking it up and having a go, but I just need more time. I'm well out of practice.
Beam, huge credibility, confirmation of working consensus and trust.
Exactly.
Are we looking at the public test before the end of the year realistically?
Yes, we are.
It's this year.
And I ain't doing it next year.
Even if we have to go without third parties initially.
It's time.
So, here we go.
We've been running 2.5 million, 2.6 million for i don't know 10 minutes
maybe um we have see what see this it's stable where it is but if i push any more through it
this really starts to get nasty this goes through the roof and things start to go wrong i mean it's not a bad problem
to have things going wrong at two and a half million a second but there is more um so that
i think concludes my very late in the evening demo the bar's been pushed i don't know anybody that's done this in a test not a legitimate
test with geo distributed validators lots of them everything turned on verification validation
consensus all those things nobody's done this nobody's even close to this on anything. So basically doubled the past, the last one,
and then we'll double it again. So yeah, hopefully that was somewhat informative to you guys.
We'll do this in a couple of days with the community group. I'm going to bring some more
people into that as well, by the way. It's time some more people into that as well by the way um it's time
some more people came in and got familiar with setting up cassandra nodes and running them and
stuff because you'll need all that experience for um the testing that's coming along very soon
move the validate move the spam away from the validators and then we'll shoot
for another big number be nice if we could get 5 million out of a 32 shard group
that'd be incredible and then we can bump it to 64 which is where i kind of want to run the public
test net at based on what i think will come in compute wise that will give a nice healthy network
in terms of potential performance but also decentralization show some of the you know you'll have
byzantine tolerance to some degree so 64 shard group doing 10 million transactions per second
and a couple of million swaps per second would actually be a pretty big win so i'm gonna sign off and get to bed because it's been an 18 hour day again um she'll
spread the word let everybody know that scalability they don't have it especially compared to this
and i'll catch you all soon thanks thanks for joining
sign in next time i am definitely into bed.
So tired.
Night, everybody.
