Pushing genesis back to 0

Technical questions about the Ripple API, the protocol, etc.
Google [Bot] like(s) this thread.

Re: Pushing genesis back to 0

Postby donch » Fri May 23, 2014 1:32 am

lucky331 wrote:i wish this gets resolved soon. i do not want chris larsen on stage being asked questions about this. this will raise doubts and some might even tag ripple as "broken" and "flawed". i wish there was something i could do to help.

what can the community do? let's all pitch in.


Without the transactions and their original signatures, there is no chance of reconstructing the ledgers at all. It's a 256 bit cryptographic impossibility without quantum computers :-)

Basically the ledger hash takes into account the hashes of all the leaf nodes which itself takes into account a hash of the signature which is itself cryptographically secure. No transaction and its original signature, no ledger.
donch
 
Posts: 796
Joined: Mon Nov 18, 2013 8:07 pm

Re: Pushing genesis back to 0

Postby lucky331 » Fri May 23, 2014 1:36 am

i suggest RL get a spin doctor ready.
User avatar
lucky331
 
Posts: 2729
Joined: Sun Jun 02, 2013 11:22 am
Location: got jed?

Re: Pushing genesis back to 0

Postby AndrewSF » Sat May 24, 2014 2:47 am

I learned today that their is a dedicated engineer working on getting the ledger header issue fixed.
Ripple user since January 2013.
Donating is good karma! r3CxMDQFX1Atq1vQN5sqVsjm3rawGSwd32
http://onemilliongateways.com/ - Join the Ripple Business Association at http://www.xrpga.org/
AndrewSF
 
Posts: 300
Joined: Fri Feb 15, 2013 7:06 pm

Re: Pushing genesis back to 0

Postby donch » Sat May 24, 2014 3:00 am

Has someone found the transactions?
donch
 
Posts: 796
Joined: Mon Nov 18, 2013 8:07 pm

Re: Pushing genesis back to 0

Postby Sukrim » Sun Jun 29, 2014 11:04 am

Well, nothing much being posted here in the past month... so I decided to go ahead a little bit and at least preserve what we already have out in the wild, namely Wisepass.com data.

Here is a zip file with the html source of wisepass pages for https://wisepass.com/ledger/0 until https://wisepass.com/ledger/32579 (inclusive). They have been prettyfied by BeautifulSoup 4 (to make them maybe a bit more readable and maaaaybe even easier to parse depending on what you use).

http://www.wuala.com/Sukrim/Public/Ripp ... /wisepass/

My next step is to extract the little data that is in there (Ledger header hashes, transaction hashes, header timestamps) and discard the probably wrong data (some timestamps are in the future of 32570 for example) to have at least a reasonable base for data. Especially useful is the fact that we know of transactions that took place in older ledgers since they are referenced in 32570 to test for bullshit data. Also there are LedgerHashes entries in 32570 (and feven later hashes) that could be useful to at least extract some ledger header hashes to verify wisepass data.

Maybe another question that might be better suited to donch, JoelKatz etc. - how exactly do I create the TransactionRoot hash if I have a transaction hash and a (serialized) transaction (if needed)? Looking at https://ripple.com/wiki/Ledger_Format#Tree_Formats it is not very clear to me how to deal with the (many) single-TX ledgers and my shot in the dark with just SHA512 hashing the txid did not lead to useful results.

Last but not least, we should be able to at the very least reconstruct AccountRoot hashes for ledgers back to 32301 (when 2E30F12CCD06E57502C5C9E834CA8834935B7DE14C79DA9ABE72E9D508BCBCB1 was applied) since then there were no transactions until 32570, so it should have been purely deterministic. Unfortunately Wisepass does not seem to have any useful data back in that window but LedgerHashes might(?) help.

Stuff that could be recovered with wisepass data:
fewer degrees of freedom for header time stamps (Close Time and ParentCloseTime)
maaaaaybe (if txid is enough?!) some TransactionHash entries
some PreviousLedger hashes

Still missing: lots of StateHash entries (these also depend on transaction signatures afaik, so they are impossible to recover if Wisepass didn't store the hashes or if old transactions stay lost), only ones between 32302 and 32569 are deterministic with what we know (if we can recover/find/guess a few crucial LedgerHashes entries) and likely some TransactionHash entries too (we don't even know all old TXIDs).
Sukrim
 
Posts: 1826
Joined: Mon May 20, 2013 10:44 am

Re: Pushing genesis back to 0

Postby donch » Sun Jun 29, 2014 11:35 am

Sukrim wrote:Maybe another question that might be better suited to donch, JoelKatz etc. - how exactly do I create the TransactionRoot hash if I have a transaction hash and a (serialized) transaction (if needed)? Looking at https://ripple.com/wiki/Ledger_Format#Tree_Formats it is not very clear to me how to deal with the (many) single-TX ledgers and my shot in the dark with just SHA512 hashing the txid did not lead to useful results.

The TransactionHash is the first 32 bytes of the SHA 512 of 0x4D494E00 plus the binary form of the root inner node. For a ledger with few transactions, all of which do not have the same first nibble of the hash, the inner node is the 16 hashes of the transactions, not the transaction ids, some of which will usually be zeroes. For a Transaction Node with Metadata, the hash is the first 32 bytes of the SHA512 of the format described here:

https://godoc.org/github.com/donovanhide/ripple/data

For ledgers with many transactions a tree will result, at which point it gets even more complicated :-)

I'm working on some code that will enable the reconstruction of transaction and account state trees here:

https://github.com/donovanhide/ripple/t ... ter/ledger

But it's incomplete.

In terms of using the vestigial Wisepass data to try and get some more info on the missing ledgers, I believe it will be in vain. All the entropy is contained within the transaction signature. Without these complete transactions, there's no hope of matching any hashes. Which is annoying, beyond belief :-)
donch
 
Posts: 796
Joined: Mon Nov 18, 2013 8:07 pm

Re: Pushing genesis back to 0

Postby Sukrim » Sun Jun 29, 2014 12:26 pm

Interestingly wisepass only misses data from ledgers 0-60 and from 32272-32569, everything(!) else at the very least has reasonable timestamps, I'm checking right now for hashes we know about (contained in 32570). The early ones (0-60) have timestamps in August 2013, the 32... ones have timestamps around January 4th 2013. This could at the very least help with reconstructing LedgerHashes entries.

Edit:
The "256" skiplist (the shorter one) from 32570 seems to be mirrored in wisepass data, except for the last one, 32512 (which is also expected). It has every 256th ledger, starting from 256, then 512 etc.

Interestingly the "2" skiplist (the long one) does not match to ANY ledger hash in wisepass at all, only the last hash (60A01EB...) is the one for 32569, as it is also stated in the header of 32570.

Next step is now checking for TXIDs to see if wisepass matches the data in 32570 and if it maybe even contains additional TXIDs.
Sukrim
 
Posts: 1826
Joined: Mon May 20, 2013 10:44 am

Re: Pushing genesis back to 0

Postby Vinnie » Wed Jul 02, 2014 8:56 pm

I really appreciate the amazing efforts that are going into the reconstruction of this data!
Vinnie
Ripple
Ripple
 
Posts: 212
Joined: Sat Feb 16, 2013 1:04 am

Re: Pushing genesis back to 0

Postby donch » Wed Jul 02, 2014 9:07 pm

Vinnie wrote:I really appreciate the amazing efforts that are going into the reconstruction of this data!


Hi Vinne, the following quote appears a few posts back.

AndrewSF wrote:I learned today that their is a dedicated engineer working on getting the ledger header issue fixed.


Is this correct? To be honest I've given up trying to salvage anything from the data that we were given. It appears to be a dump of the wrong databases. Without the original transactions and, most importantly, their signatures, I see no hope of ever matching a single ledger hash. The skiplists do contain the ledger hashes, and they can probably be ordered, but without an accurate root TransactionHash (which depends on the signatures of the transactions in that ledger) there seems to be no point in trying.
donch
 
Posts: 796
Joined: Mon Nov 18, 2013 8:07 pm

Re: Pushing genesis back to 0

Postby Sukrim » Thu Jul 03, 2014 8:43 am

I agree to donch, while it might be remotely possible to reconstruct the content of some transactions (that we know of in 32570) and the data dump by JoelKatz might help there too, in case Ripple Labs did some "standard" transactions every time they restarted a new ledger chain, without the original data there is practically 0(!) chance to recover all of the initial distribution, especially in a format that can be re-imported to rippled. Never ever in our lifetime and the lifetime of humanity on earth.

Who is working on this internally, could we get (maybe via PM) a mail address or skype contact or anything?
Since I trust JoelKatz, it looks like this data is definitely not at RippleLabs any more and lost there, however for example the data from Wisepass at least looks very promising. Has Ripple Labs contacted them and others who ran rippled very early on (this is something that we outside of RL can NOT do as we don't know) to ask for database dumps?
Sukrim
 
Posts: 1826
Joined: Mon May 20, 2013 10:44 am

PreviousNext

Return to Developers

Who is online

Users browsing this forum: No registered users and 3 guests
cron