Page 2 of 14

Re: Pushing genesis back to 0

PostPosted: Fri Jan 03, 2014 4:55 pm
by ahbritto
donch wrote:I do think this needs to be done - without the genesis ledger being viewable and the initial distribution of XRP being auditable, it is hard for Ripple to be 100% credible...

Do you think people do not find you 100% credible as they don't know your complete ancestry? ;)

Re: Pushing genesis back to 0

PostPosted: Fri Jan 03, 2014 5:05 pm
by donch
Sukrim wrote:There are already a few earlier transaction hashes known (they are referenced in ledger 32570), however there are still a few that are not easily reconstructed - and since the signature part of them should better not be bruteforceable (else we'd have a problem!) there is no real way of reconstructing history by just guessing how each transaction might have looked like.


You might be getting confused between the transaction hash, which is the key in the massive key value store of the parts which make up the state tree and the transaction signature, which is generated at time of submission and is encoded within the transaction itself. The hash is easily reconstructed and there is no need to brute-force the transaction signature, it already exists!

It really depends if LedgerSeq was in the Transactions sqlite db schema in the code base at that time.

Sukrim wrote:Even if I get the ordering and values of transactions completely right, as far as I understand it the hashes of these transactions also depend on the signature of the sender and this means that it is currently impossible to bruteforce. Without the signatures on these transactions there is nothing you can realistically do, ideally JoelKatz would either work on his secret stuff on his own and just push it to the RippleLabs servers soon or finally publish the database dump or whatever he seems to have of these old transaction and node data for others to try. Once I have the transactions, I can try the combinatorial approach, as suggested already (ordering should be quite clear anyways, it's mostly a matter of which ledger has which transactions + account state). Until then - 0 chance.


The only signature that needs to be generated is a validation one and that could be done by a new private node key or one for the one of the existing nodes, doesn't really matter.

Sukrim wrote:Rippled (dev branch) is getting better in fetching history reasonably fast by the way! :)
You might wanna try to compile and run a bleeding edge version. Unfortunately my Windows builds still error out after a while and I was still not able to really debug them (my Linux machines are too slow I/O wise for rippled and I dislike VisualStudio, also I'm not too sure what the problem really is and how to debug an application that randomly crashes inconsistently, sometimes after hours or days).


I'm using the develop branch from github and I still only have"2262330-4228407" after over a month and the disk usage so far is 162GB! My main issue is that I just want all the transactions and ledger headers and these are not readily available. My code will scan forwards from 32570 (or 0 one day) perhaps checkpointing along the way. Because of the design decision to permit syncing backwards through the ledger the whole of the huge state tree seems to be required. The actual transactions and ledgers only take up a small fraction of that space. I don't want my thin client to work that way :-)

Re: Pushing genesis back to 0

PostPosted: Fri Jan 03, 2014 5:07 pm
by donch
ahbritto wrote:
donch wrote:I do think this needs to be done - without the genesis ledger being viewable and the initial distribution of XRP being auditable, it is hard for Ripple to be 100% credible...

Do you think people do not find you 100% credible as they don't know your complete ancestry? ;)


Well, Ripple is meant to be an open ledger, and we can't currently view the times and amounts of the initial major distributions of XRP...

Imagine if some of the Bitcoin blockchain went missing :-)

Re: Pushing genesis back to 0

PostPosted: Fri Jan 03, 2014 10:12 pm
by donch
ahbritto wrote:
donch wrote:I do think this needs to be done - without the genesis ledger being viewable and the initial distribution of XRP being auditable, it is hard for Ripple to be 100% credible...

Do you think people do not find you 100% credible as they don't know your complete ancestry? ;)


I think a better analogy is having a child but then losing the birth certificate ;)

Re: Pushing genesis back to 0

PostPosted: Fri Jan 03, 2014 11:10 pm
by Sukrim
donch wrote:
ahbritto wrote:
donch wrote:I do think this needs to be done - without the genesis ledger being viewable and the initial distribution of XRP being auditable, it is hard for Ripple to be 100% credible...

Do you think people do not find you 100% credible as they don't know your complete ancestry? ;)


I think a better analogy is having a child but then losing the birth certificate ;)

Or trying to do an audit without any documentation for some transactions...

However as JoelKatz said, he anyways still has (hopefully) all the necessary things to put this together - all he'd need probably is time to actually do it. That's why I asked to have access to this data as well, since I would have had that time. I can understand not having time to work on this issue, I don't understand not having the time to even post the data itself, there might be something I'm not thinking of or maybe the stuff is buried deep in huge databases or encoded in chinese or whatever else that makes it difficult to access. Still I hope that soon (TM) something happens that allows people to have a look at ALL transactions in Ripple, an not all since 32570.

Re: Pushing genesis back to 0

PostPosted: Fri Jan 03, 2014 11:20 pm
by donch
Sukrim wrote:Or trying to do an audit without any documentation for some transactions...

However as JoelKatz said, he anyways still has (hopefully) all the necessary things to put this together - all he'd need probably is time to actually do it. That's why I asked to have access to this data as well, since I would have had that time. I can understand not having time to work on this issue, I don't understand not having the time to even post the data itself, there might be something I'm not thinking of or maybe the stuff is buried deep in huge databases or encoded in chinese or whatever else that makes it difficult to access. Still I hope that soon (TM) something happens that allows people to have a look at ALL transactions in Ripple, an not all since 32570.


I definitely want the data too :-) Even the new ripplecharts.com is bootstrapping of the mythical 32570 ledger:

https://github.com/ripple/ripplecharts/ ... _full.json

Ripple Labs definitely need to embrace third party development more by helping coders get access to the data. It only takes a couple of minutes to upload a sqlite dump to S3 and a torrent file of a RocksDB directory could easily be made....

Sorry if I sound like I'm grumbling :D

Re: Pushing genesis back to 0

PostPosted: Thu Feb 13, 2014 4:36 pm
by Sukrim
Another month, another try:

Please release the transaction data for ledgers 0-32569!

Re: Pushing genesis back to 0

PostPosted: Mon Mar 03, 2014 4:30 am
by Sukrim
Aaaand March is here, the month of spring and Oscar ceremonies!

In other news: Only 3 months left until a full year has passed on the statement from https://bitcointalk.org/index.php?topic ... msg2352658:
All that was lost was the ledger headers -- all the transactions are still available


I tried querying for the TXIDs referenced in ledger 32570 to no avail, so these are at least not available on the public facing servers that I reached... maybe load balancing is screwing with me (it has done so several times...) and I just reach "juvenile" servers by accident, it would be just MUCH easier if there were a database dump available of e.g. transaction.db and ledger.db or anything really that would help finally restoring genesis back to 0!

Re: Pushing genesis back to 0

PostPosted: Mon Mar 03, 2014 4:35 am
by lukecyca
+1

Re: Pushing genesis back to 0

PostPosted: Mon Mar 03, 2014 12:15 pm
by donch
+2!

Please release the <32570 transactions with metadata in binary form as soon as possible! Ideally they'd be a way to stream all the binary transactions with metadata per ledger via the API even if the Ledger Header is missing...

If not, for this project to be truly open, you need to share what you have.