hi everyone Charles Hoskinson here I just wanted to go and make a video before Christmas about a few things related to Shelly and new features we anticipate seeing during q1 things that are priorities things that are going to be changing for the new year and anyway I figured why not so the one point four is out people have downloaded one point four it's been quite a bit to get through just a few things about one point four things that I think are important to mention one point four requires people to restore into the data layer of the system it also takes the current way that we were storing files where every blog had its own file which was a kind of a bad design and it compresses that down to basically an epic package now these two processes are very i/o intensive so you don't have to redownload the blockchain if you're not connected to the Internet still it's going to work the the reality is however that while you're restoring into the new data layer and while you're cleaning up your hard drive you're cleaning up that that the storage going from these many many many blocks and many different files down to a new data layer if you have a mechanical hard drive a non SSD or if you have an older system because this is such an i/o intensive process generally it takes quite a bit of time so for some people we've seen restoration rates in the hours - - some cases days but most users are able to do this in between thirty minutes to two hours some things that could be slowing down your restoration time could be your antivirus software and to fire software's our heuristic engines and basically what happens is if they see something all of a sudden taking a huge amount of i/o or if they see lots of files moving around they say oh this is suspicious behavior and they tend to analyze that pattern against known signatures or our bad patterns now even if it clears and avy is okay with this the very act of analysis could quadruple or in some cases even by a factor of ten slow down your your time it takes to install 1.4 so if you're having those issues one thing you can do to try to speed it up is to disable temporarily your antivirus during the installation process and restoration process and then re-enable it after the process is over another thing is that you won't instantaneously see reduction in storage size as soon as the installation is complete there is a background process that runs once the restoration is done which gradually cleans up the old files and over time usually a day basically they'll purge all those old files and then you'll just have the new database and the new data layer and for many cases you should be looking at probably a factor of 10 reduction of the amount of storage on disk it depends on the file system you're using if your linux mac or windows and elyda of a little features but it's a considerable improvement and people post upgrade should really notice a big improvement there if you're having issues submit a forum request send us a bug request and we're gonna probably set up a dedicated channel on telegram for people who are just having issues they can't get it done for whatever reason to talk with our people and get people through this very large upgrade but it's it's slow unfortunately because it's a big update and you're basically moving from the old broken not so good data layer to the new awesome wonderful great data layer but in the process of that transition it does take a bit of time so be patient and understand that this is not a event that's going to happen every time we do an update this was the big one and we spent quite a bit of time building this software and we're quite happy with it it just takes time to migrate to it ok moving into q1 let's talk about new features coming to cardenal and let's talk about the shellye release so basically what is the shellye release well there are three dimensions shellie release there is a testing dimension there is a deployment dimension and then there's a social dimension so in the testing dimension we have a family of new protocols we've constructed new Network protocol or or as Genesis the delegation scheme and the incentive schemes that we've come up with has alongside this concept of state pools and all the logic behind them so basically because we have all these new protocols those protocols have to be fully implemented and they do have to have some burn in time to verify that the protocols are correct now we've written prototypes now of our network stack prototypes now for Boris EFT Kraus and Genesis and we've written prototypes of some of the delegation mechanics as well as the incentive scheme so we have a really good concept of what the new legend rules need to look like what delegation looks like we have 50 plus pages of specifications for that and those are growing and a really good concept of what Genesis looks like so the first half of q1 is about getting ourselves to a point where we can begin broad scale testing and verification that our ideas match reality and we have because of the process we fall we're feeling pretty confident that they do but that's a process and something get done the good news is that the code quality is really high the engineers are really good and everybody's roughly on the same page about what we need to do the other good news is we've dearest it a little bit by having two competing teams trying to do the same thing using different tools techniques and processes one is following a more pragmatic Silicon Valley standard that's the rest team or they're moving very quickly and then the other is following more disciplined specification driven development as a larger team but they're moving at a slightly slower pace and so both teams are working towards getting these protocols implemented and tested now a large component of that is the thing I mentioned the socialization of it so once we're ready for a test net then we have to start building a large population of stake pools out of curiosity last year we ran a registration list just to see how many people would be interested in running a stateful because we had to set kind of the parameter of what we felt would be a good target hundred to one hundred five hundred and so forth we receive I think over four thousand people who are interested in running state pools and I think we'll probably get equivalent numbers once we open that again so sometime in January our comms Department in coordination with the newly reinvigorated Cardinal Foundation will reopen that registration and we're going to try to get as many people as possible interested and potentially running a state pool and then very short order we're going to roll them over into a test net and try to get as close to that number of thousand as we can and get some to a feedback on the use protocol and then there's the actual rollover of what we have today the Byron system to where we would like to go Shelley now this means that we're going to have to issue a series of updates to the Cardinal client at least two updates will be required one is that the current implementation of or borås this or wors classic is very old implementation it's based on the ideas we had back in 2017 but we've evolved quite a bit in our design of protocols so what or bourse Genesis looks like versus what we have in our repo very different things so there's going to have to be at least one fork to get us to Genesis and so what we've decided to do is use a halfway house solution where we implement or Boris BFT we bring that in at the base level and we fork to that and then we have a migration path from bft to Genesis so BFT the legacy team's been working on that and it's the last piece of code we're going to be committing to the legacy repo the implementation of EFT is done and we've begun broad scale testing of it and we're gonna actually test it both on the rough side as well as the Haskell side we're gonna run arrest clients against the Haskell clients it's really exciting that we have two independent implementations and these implementations are running against each other verifying that we've correctly implemented from the specification that was written so once Genesis once BFT is out that will be one point five the very next thing to do is to then begin a series of upgrades to get us to basically to Shelly so there's going to be some changes to the ledger rules there's going to be some changes to the api's and there's going to be some changes to the network stack and all of these things are required for us were basically Shelley logic and you'll see that post 1.5 coming through we've made considerable improvements to our release process we've made considerable improvements to our QA process and we're now starting to work with code bases that have very little technical debt and are based upon good design principles and good engineering practices one of the reasons why we had so much trouble throughout 2018 actually getting good products out is that we were dealing with a code base that did not have good documentation we were dealing with a code base that was laden with a lot of technical debt and we were dealing with a code base for the original engineers who wrote that were no longer at i/o HK and they left so we were basically taking a halfway house of a halfway house that was not well understood and we were trying to clean it up and make it reasonable and if you look at the progression of updates 1.1 1.2 1.3 at 1.4 we mostly accomplished that this year to the extent that we completely rewrote the entire wallet logic we also heavily refactor the core so that we could actually write tests against the core and we wrote retro actively a specification for the core which is now finding its way into our new ledger rule specification so we actually had to write two formal specs one for the old system and one for the upcoming new system the old system basically is required because we still need to validate legacy transactions as we move to chalet so that's all been done but it was a heck of a lot of work the good news is the new system doesn't have any of these legacy concerns we designed it we wrote it and it's based on good principles so it's much simpler and it's much easier to test it's much easier to understand it's much easier to keep in our head furthermore we now have a lingua franca for our bow basically design we say here's the set theory here's the math you can read a spec it has some verbosity to it there's some words so she ate with the math and this is what Shelley is so the core of the system is an implementation agnostic collection of mathematical representations and any developers should be able to take that read it understand it and say ah I understand what you're talking about when you say this is a valid transaction versus not I can reason with the spec to understand one versus the other from the UT Exocet to the ledger rules to how consensus will work the how the network will work that is the sole of our protocol now that we have that we're moving along quite well so basically you're going to see one point five B or bors B of T you'll give everybody a heads-up about that and we'll have a much more graceful upgrade path it won't require restoration it won't require a new data layer it won't require you to do much it should be like a normal upgrade so it won't be as painful as one point four is but that'll get done and then after that that we will have a decoupled wallet back end will have icarus style addresses and then we're just going to gradually move our way to the v2 API and gradually move our way to all of the things that are required for Shelly and then they'll likely be one more up major update and then we'll put in all the infrastructure and then Shelly will just slide in now some threshold conditions for Shelly are it's not just good enough to have all the code written and it's not just good enough to have all of the protocols ready to go and fully tested it is a social component as well we do need a certain critical threshold of people who are actually deploying state pools running state pools and understand what that means so that'll be done in a very visible transparent way and what will likely happen is that we'll have a gradual winnowing of federated control to decentralize control where epic by epic basically more and more of the slots are made by state pools and less and less the slots are made by the or Boris B of T and that eventually that just disintegrates and a hundred percent of the slots are made by state rules following the protocol so it's as if the epochs are getting larger for decentralisation so you'll see a series of videos in blog posts and announcements and Duncan will go and talk about this as well others but progress is pretty good now in terms of the in terms of the feature set now that we have a decoupled wallet back-end coming January what's really exciting is we can start augmenting and really taking that wallet layer to the next level so if you look at the rest client one of the cool things you can do with the rest client is that you could manually construct transactions eventually you can sign offline transactions manual you TXO selection and this idea of a wallet scriptlet so basically it's a small script you can write like a shell script or you know is like a Python script something that you can write that automates the wallet process now you can do that with the RUS client you can't currently do that with a Haskell client big priority in q1 for the Daedalus team will be putting a terminal into Daedalus itself and then allow you through that terminal to interact with wallet layer and giving you the ability to have an advanced user mode where you can manually construct transactions do offline signing do you TXO selection these types of things in addition to running your own scripts inside the system and eventually coming up with a clever way of packaging them and deploying them so we can talk about automated wallets this is just a really cool thing to do a lot of power users want to do it for a variety of reasons for forensics reasons two other reasons the exchanges wanting to run these types of things and so it's a feature that we're going to start designing out and gradually rolling in another thing you guys are going to start seeing is we're going to start writing some content about what the delegation center is going to look like what the delegation process is going to look like and also the presentation of things like steak pools this is a slightly controversial topic because just like if your Google Play or iOS or any of these app stores he who controls what you see first just like a search engine if what's on the first page is usually the page that wins out if you're on the second page of the third page or if you're in a less prominent area of the GUI you have a significantly lower percentage chance of getting eyeballs attention and interaction so it's very important that we have a discussion in the community about what is the optimal way of displaying the the delegation rules that are available and and how those mechanics will work so what we'll probably do is once the scamp's are fully done and that'll be in early January to mid January for a first set of them I will start socializing that and then we'll probably go to community groups like the Cardinal effect for example and do a whole presentation like we did with episode 8 talking about the 1.4 release and getting some community feedback on different ways we could potentially do this and eventually converge to a good design that we think is fair for people but it is an important point to say that this is now dwelling from hard core mathematics and protocol design into opinion and into what we think is fair so this has to be socialized and go beyond just our best guess so look for that as well so terminals coming that's coming now we had get asked a lot about ledger support now that we're on the new data layer we're gonna have Icarus style addresses which are easy to 1529 bib 44 compliant and ledger does support that and what we've done is we through amer go have begun discussions with a firm called vacuum labs they did a very serious analysis of where the current ledger code was at that was done by the Cardinal foundations prior contractor and they've come up with an upgrade plan to take that base and basically make it compatible with our system and so what we're going to do is we're going to engage them and they think around five or six weeks worth of engineering work is required starting in early January and that work will once done should give us the ability to easily interact with ledger and if that's the case that ledger support should come in short order so the soonest you could expect that on the Dedalus side would be February but we potentially might be able to get it out a little earlier on the arroyo side your ROI is also working on treasury support as well so you'll notice that once a crystal addresses are supported by Daedalus there's going to be another transition where we'd like to get people from the old nondeterministic addresses to the new Icarus addresses but because you'll have significant faster recovery time for wallet restoration for the more the address size is about half the size so easier dresses to work with faster recovery time just overall better experience for people furthermore those will be the addresses supported by the Leger devices not the old Daedalus style monitor mystic dresses for the more than the icarus addresses and the Daedalus addresses will be interoperable with each other so your ROI and Daedalus should be able to restore from each other at some point so we're still in coordinating with Nikko and Sebastian about that but that's where we're at I'd like obviously to wave a magic wand and have these things faster but you know there was just a lot of bottlenecks that prevented us from getting there one is we had to resurrect the project from bad product management from a different organization - we had to be ready to integrate something like this and one-point-four was really the milestone for it it made no sense to attempt to get legislate alaior the old wallet because that was a very poorly written piece of code that had a lot of problems with it and resurrecting that code to a new system made a lot more sense than trying to make it work but they'd have to throw it all the way and rewrite it so sorry it took so long but now we're on good track vacuum labs good company they have a lot of wonderful people and it's real exciting to see that now another thing is that we're gonna start building GUI scamp's for the offline Center for basically its segregate your hot wallets from your cold wallets so we already have paper wallets and soon we'll have ledger support what that functionally means is now Daedalus needs to account for two different types of wallets in the system right now we have a monolithic system where there's one type of wallet and you just create as many different accounts as you want within that one wallet class so each and every one of them is an HD wallet but at some point you're gonna have now your paper wallet and your ledger device and your regular hot wallet now the reason why we need to segregate these things is that there's this concept of cold staking where you have your ledger device or your paper wallet and you want to be able to control the stake for the purposes of delegating and eventually voting on corrado improvement proposals or Treasury ballots but you do not want to take the private keys and expose those private keys to a hot wallet you don't want them living on disk living on network you want to keep them on paper or keep them in that hardware device so that they're safe and secure so there needs to be a way from a GUI to see what your balance is on that device and have a process where you can import a transaction giving that state key authorization or import and memory only these credentials and then transfer them and give control of them to Dedalus but keep the keep the spending keys offline so basically we need to build a Center for that and that's in parallel something that the Daedalus team is working on we've already started GUI discussions and they'll converge probably about mid January to February and it's easy to do these things it's just we need a few more things and data lists to make that happen but the good news about Daedalus is that it uses a really easy to understand a GUI approach we use react polymorph and it's built on top of chromium and node because it's using the electron system so we're using a very traditional web style model and as a consequence we can do very rapid prototyping very rapid iteration when we're doing these camps sometimes a matter of days to weeks so converging to a design is actually quite straightforward it takes a lot more time to actually build that functionality on the back end but the new wallet layer is much more recently robust and easier to extend than the old model so we should be able to get new features out so in summary Shelley has three components a social component a testing component and an upgrade component upgrade taking the existing system and gradually morphing it into the Shelley system testing meaning that we have to verify that our version of reality is correct and we think it is but we have to absolutely make sure it is and we're getting close to being able to do that and the social component meaning we have to build up on a large pool of state pools a large amount of state pools we have to get people used to the delegation Center and get people comfortable with that presentation how all these delegation mechanics are going to work and basically just set a date once all these things are done we've hit certain thresholds and then once we flip the switch they'll be gradual reduction of bft nodes that control the system and epoch by epoch down to basically a complete decentralisation so it won't be instantaneous it'll be gradual epic by epic and then we'll just wake up maybe 10 epochs later and suddenly the entire system's totally decentralized so that's a little bit about Shelly Ledger's coming we now have a great third party partner for that your ROI is evolving at a great rate in fact I think Sebastian eco are gonna have some great announcements here in a little bit and they're really doing a wonderful job of making cardano's easy to use and that's really exciting now as a final point let's talk about usability I'm really really excited about the prospect of actually making Cardinal a good user experience you see guys the problem with our ecosystem has always been that people have entered into Cardinal with unrealistic expectations and those unrealistic expectations were that Cardno and Daedalus are consumer products that are completely polished and finished like Windows 10 or your latest version of Mac OS this is not true when iran was released in 2017 it was a very early product and it was a huge learning experience for our users and ourselves and if you ever use beta software if you ever use version 1 software of some completely new thing using completely new techniques the reality is you're gonna have bugs you're gonna have frustrations you're gonna have unexpected crashes you're gonna have all kinds of boneheaded things that come up and that's just a fact of life so you know that it's I empathize and understand that people have had issues like connecting to network or things are slower than they ought to be or you know things take too much memory or disk space but this is just growing pains and what you'll notice is that over the course of this year we have made massive improvements to the system the network stack has gotten three to four times faster on average memory consumption for Dedalus has gone down by 75% if not more dis storage has gone from multi gigabytes to hundreds of megabytes to a megabyte or gigabyte or two we've seen a massive improvement in UTX so selection and now we have a system that will not fragment whereas the old system did and once we move to deterministic addresses will have paths to a now Nabal very quick recoveries like Icarus did furthermore we were able in a very short period of time to build a great user experience with chrome and hand it off as a reference product to infinito and to emerge oh and in both cases they were able to get things that have now enabled to hunt thousands of Cardinal users to be able to use our protocol comfortably with a pretty good user experience so it is a beta product we've been working real hard to make it better and there are massive wins throughout this year that demonstrate that and it's also very important to point out that well we've not had massive or significant bugs such that huge sums of money have been lost such that the entire network went down for weeks at a time such that the vast majority of the users the upon just doesn't work and we also have had multi-platform support Windows Mac and now Linux 1.4 is on Linux so what we're gonna do for 2019 is I which K is now getting to the point where you feel it's worthwhile to have a dedicated product experience officer and that person's role will be to start with the user experience and then work the way backward to exactly what we need to do to enhance our protocol and technology to massively improve certain things like startup time improve usability redo the UI and UX these types of things it wasn't quite reasonable I'd say in the first year because this was basically a big experiment and there was a huge amount of learning on our organization on how to do QA helpdesk how to get the release cycles where they needed to be working with exchange is working with enterprise users working with everyday consumers but we have proper segregation now we have proper processes were in a position where such a product experienced officer does make sense so that role will be soon put into our HR queue and it's something that throughout 2019 we're going to worry a lot about everything from the unique experiences of delegation to how we want to roll out multi-sig to how we go about improving basic things like speed for example and what trade-offs are we willing to make for that for example if you want to do a fast restoration under a deterministic HD wallet if you have access to an explorer you can do a restoration at a very fast pace less than a minute in some cases as we've seen with your ROI but that doesn't mean you do potentially have to trust some third party if you don't trust anyone the best you can hope for is a restoration in the 30 minute to an hour range depending upon the speed of the PC and exactly how things are stored and so forth so you know this is just a reality of software in the reality where we live another thing is not everybody needs to have a full client when we first launched it it made sense to build up a large node population of full nodes so that when the full peer-to-peer switches a lot of the full networks which is not we have a resilient backbone of full nodes but at some point we do need to see client segregation on the light client side as well as the mobile side and that's really what 2019 is about so we'll see a natural settling of some people preserving those nodes still operating them some people switching over to lighter experiences and some people switching over cell phones so it's been a productive year we've certainly learned a lot just simply creating a lingua franca between business people engineers and scientists was a pretty Herculean task but it was accomplished and we have a lot of workshops coming and we have a lot of things coming out a lot of socialization coming out we will make best effort to deliver in q1 of 2019 and unless there's a significant delay we'll see what happens but it's it's always been bold to try to get things out when they can but also make sure that they come out with reasonable quality and they come out with a good user experience and they come out with a high ability of success it's incredibly important to understand that when Shelley comes out cardano's basically on same footing with every other major cryptocurrency in the space in that it is fully decentralized we'll be fifty to a hundred times more decentralized in aetherium Bitcoin and AOS in terms of the control dynamics that's a that's a very big deal and if you don't have the right social structures in place for that it won't be successful so that's a huge priority now that the Foundation's back into the fold and they're operating again as they wake up it's something they're going to begin investing money into and it's something we as an organization care a lot about and our ecosystem cares a lot about and I'm just really glad to see that what we're doing is being met in tandem with great work on the community development progress so we'll see the card on all Ambassador Program running out soon we'll make some sort of announcement about the hub program and the card oh no effect has been growing at a pretty great rate and it'd be wonderful to see copycat podcasts and different languages in particular Korean and Japanese running and I think over arc of time that we can see these things stabilized and just become the norm in our ecosystem so brief 30 minute update thank you guys all for listening and it's been quite a bit of it's been quite a bit of a ride and Merry Christmas and a Happy New Year and I hope to see everybody soon in January Cheers
