5 Shards: Protocol Update

Recorded: March 14, 2024 Duration: 0:20:49

Player

Snippets

thank you all so much for tuning in to today's Twitter spaces
today's Twitter spaces is essentially just about the recent near protocol update
should be a pretty quick one today we probably caught it at the 30-minute
mark but we just want to make sure that everybody is kind of just debriefed
updated and we have a couple of questions and topics we'll go over today we are
joined by two folks from Deepagoda team we have Bowen and Jim with us today I'm
Marcus from near foundation so before we dive into some of the questions and
just quick updates Bowen Jim why don't you guys go ahead and introduce
yourselves yeah hey everyone Bowen head of protocol and yeah happy to be here today
talk about the other recent updates about the protocol and I'll answer any
questions people have awesome thanks so much I know Jim is currently experiencing
a blizzard unfortunately where he is so I am trying to get him up here to to
speak perfect
amazing thank you Jim thank you Bowen maybe it's on my side I'm having some
general Twitter spaces issues which we've all definitely ran into where
see Bowen's a speaker but I can't see Jim is a speaker but you're here so
that's amazing so as we as we all know there's been a fifth shard added right so
a fifth shard has gone live absolutely amazing awesome developments before we
got to get into like what users can expect would love to kind of dive into
maybe what some of the major challenges were faced from the team during the
protocol upgrade and how they were addressed yeah so it's definitely not
the most smooth upgrade the problem is that when we develop the feature it was I
think sometime last year and we went to testing near did not have nearly as much
usage as it today and actually I think the the size of the the database that on
the you know RPC nodes and validator nodes group by maybe like oh like closer
how representing the past several several months and that led to this
problem that when they when the node needs to perform restarting it needs a
lot of additional disk space I think back then you know when the feature was
developed we was considered to be fine but said because it is like dramatic
grossing usage that that is actually called that caused some validators to
not have sufficient disk space so they're not crashed and they have to fix
it so that's why the during the upgrade the network was slow because some
validators need to get their nodes ready and yeah this is definitely
oversight like on our side in the development process should have
something more about what's the most recent database size and things like
this so yeah that that's not the not the most
smooth upgrade experience but we definitely got this through and the
restarting itself I think it it worked pretty well
yeah after the restarting finish I don't think anything unexpected happened
so like that protocol upgrade actually went fine it's yeah it is more like the
during the upgrade there's some hiccups
awesome yeah that's kind of see the hiccups a little bit but it was
overall it was generally smooth on the user side like
what so from the user's perspective right what can users expect in terms of
like transaction times like network performance and anything just overall
experience moving forward yeah I mean I believe none of that
experience really would really changes because like
people like users generally don't know about
shards right know about how many shards there are like on the protocol and they
don't really need to know I mean their transactions are automatically routed to
the shards that they need to be processed on
and so on so forth it just like the throughput of the network would increase
by having another shard but otherwise yeah
nothing really changes for people perfect I mean that's that's like
ultimately what we would like to hear there are you know generally growing
concerns around a major protocol upgrade you know people get a little bit
concerned on how the experience is going to be
so the fact that people generally won't really notice much
you know but the network is you know quicker more secure like that's just an
amazing kind of thing to hear however on the actual like the validator
side and node operator side you know that
that might be a little bit different so are there any kind of changes
to that side of things is there any support that's extended to them
any continual effects post upgrade going live that may still affect
validators yeah so definitely during the upgrade
they incurred a lot of additional disk space usage
and we expect that so most of that is gone
after the upgrade is down but still there's this part where
they're still keeping the state of the old shard until it's garbage collected
and that's going to take a few days so they should see
the disk space usage going down as time goes on
and yeah we are in active communication with the validators answering their
questions as well as other node operators
people who operate obviously and and so on
yeah i mean i don't know whether jim wants to talk more about this
yeah i know jim was like kind of going out in some messages but yeah if you
kind of want to publicly you know relay any of that
information here while we have the spaces ongoing jim
awesome i appreciate appreciate the information
and also what can like developers do to ensure they're
using most reliable and efficient infrastructure services
oh perfect awesome sounds good
oh perfect awesome sounds good um man i would generally love to pin some of
that information for the listeners but just keep an eye out uh some of
that stuff will probably roll out through the official channels for everybody here
um and i would like to i know bohen has a hard stop so i would like to
also open up the floor if anybody does have any general questions
um but as nir moves towards having six shards
uh can you share any insights on how this scaling strategy will evolve
uh yeah absolutely so yeah right now we are doing this uh splitting of shards uh so
this is mostly to address a problem of like too many hot contracts around on the on the
same shards meaning that there are like a lot of counter there are some contracts that people
call like uh they want to use a lot uh and they are co-located uh on the same shards so that
that for example is causing uh some pretty severe congestion on uh one of the shards for example
and uh yeah so basically previously we see this kind of contraction of both shard two and shard
three uh we finished splitting shards three into two shards and and now we're going to also
spread shard two into two shards um and uh yeah i think i think this is more like a
reaction to the uh to the level of uh usage on the network right now uh but they are looking
forward we have the status validation release uh in a few months uh so in q2 this year i think
that's going to both improve the performance of each single shard and also change the
architecture in a way that's uh that will allow us to skate scale to much more to many more shards
awesome so in the future like you know generally like high usage dApps
um that are really kind of taking a chunk of the network traffic um will it be a bit more
straightforward or seamless of a process for them to operate on their own shard um or is it still
going to be uh maybe a bit of a process yeah so so right now the process of restarting is uh
is very manual right so essentially uh the protocol developers have to calculate the boundary
at which the restarting happens and this is like a code change and needs to go to the release
process so uh so we definitely we want to move to a more uh automated process i mean this can take
several steps i mean once one step for example could be that uh there's like a smart contract
where that uh dictates the short boundary and then uh you know it can be like a dull voting to
actually change the boundary and then that actually triggers uh restarting for example and
then obviously the uh the ultimate goal is that for you know we don't even need that that
the system itself is um intelligent enough to understand uh when there's congestion that a shard
should be split into two uh and when there's uh under utilization on two different shards it
can be merged back into one so that's that's the ultimate goal that uh we have and then yeah
the point if someone wants to deploy uh there's more contract yeah you can just deploy some shards
without having to worry about um which exact shard they deploy on
amazing cool so it'll kind of move more into a dynamic system uh moving forward is the ultimate
goal it sounds like which is going to be much more effective um so i would love to i mean i
would love to open up the floor for anybody here if they have any questions about the recent
protocol update uh upgrade and maybe um you know what what to expect moving forward any
feedback etc um but before we do that uh you know feel free to come uh in request speaker but
bowen um jim do you have anything that we didn't cover uh that you'd like to address
and highlight uh for the folks here at all or any any other remarks
yeah yeah yeah sorry i forgot to talk about that um yeah so so basically uh the the hot uh
so how'd move move some of their traffic to off chain and that actually reduced the the
load on shard two right now and i think now is that uh more like normal state um and and
because of that we uh we postponed the uh the restarting of shard two into next week
uh well this is not just because of that but we also want to actually fix i have time to fix a
problem that validated experience during the the last restarting where they run off out of
disk space um and so we want to uh get an optimization in where it would uh like optimize
away most of that extra disk disk space usage so that is what we're working on now and and
that's why we we are postponing the releasing to next monday
super good to know um i just want to reiterate if anybody does have any questions about the
recent protocol upgrade um please feel free to request speaker we have uh we have a few more
minutes we can take a couple questions um dv crypto uh x has requested speaker let me go ahead
and get them up here all right you are connected hey yeah thanks for doing the space definitely
big fan of near and sharded blockchains and question is this is uh network sharding live
as well or are all the validators still validating all of the shards
uh not all the values writing all the shards so today the block producer is still validating
all the shards yeah that would change which the stillest validation uh release that's coming
uh in a few months uh and today there's chunk only producers that's only producing chunk for
uh a specific chart god okay yeah that's what i thought you said that's coming in a couple
months on yeah so we're targeting q2 this year awesome okay and with the the current
shards what are the the finality times block times intro versus cross yeah it doesn't really
so like on near like uh intrashard and intrashard communication has essentially no difference is
created basically the same uh so yeah finality is basically the same um uh so we have the
your rate got finalized within like two to three uh seconds or two to three blocks
awesome once the the network sharding is in place under the block producer sharding
how is that still going to be accomplished that cross shard times are similar to intrashard
intrashard yeah it doesn't change that part uh because what what it what it has changed is
basically the uh the improvement is mostly removing the security assumption that uh block producer
needs to track all the shard i mean today they're they're tracking that they're tracking all the
shards not for speeding up the the inter shard communication it's actually for security of the
network got it okay awesome thanks for the answers love it
awesome great questions thank you so much for coming up here uh we do have a little bit more
time so if anybody else does have any questions any any remarks anything that they want clarified
around either this recent protocol upgrade or what's coming up in the next week or two just
please go ahead and request speaker
all right it does not look like anybody else has any questions at all you know there was quite a bit
covered in today's session really just want to make sure we're continuing to update the community
regularly and transparently and addressing any any remarks that come with any of these major
upgrades so i really do appreciate the time that bowen and jim took to just come and do the spaces
but just one last time bowen or jim anything that maybe didn't get covered or that popped up
before we go that you want to say any last words
yeah as jim said um yeah it has been a busy week for us but it's great to see the
near getting more and more used more and more people wanting to use different apps from here
and yeah we're working hard to make everything uh more scalable
awesome thank you all so much for tuning in and until next time we'll see you soon take care