Pushing genesis back to 0

Technical questions about the Ripple API, the protocol, etc.
Google [Bot] like(s) this thread.

Re: Pushing genesis back to 0

Postby lucky331 » Thu May 22, 2014 4:52 am

Without these early ledgers "Trust" becomes a legitimate question. RL has a lot of super smart people. There has to be a way to recover the data.


there has to be. the human race have come so far and have made the "impossible", possible.
User avatar
lucky331
 
Posts: 2729
Joined: Sun Jun 02, 2013 11:22 am
Location: got jed?

Re: Pushing genesis back to 0

Postby Sukrim » Thu May 22, 2014 10:49 am

celticwarrior72 wrote:RL has a lot of super smart people. There has to be a way to recover the data.

Well, if they only ran on "cloud" providers, then there are no HDDs that could be pulled and scanned for file fragments. As I said some partners of OpenCoin/RippleLabs might still have some old backups or logs lying around, maybe some employees were toying around with rippled and syncing with the network?

Unfortunately it took quite a while until we got this data now, but better late than later... it is still just 1.5 years after the data loss, not 10 or 20 - so we might get lucky after all.

As I said earlier in this thread, wisepass for example seems to have some data in their "ledger explorer" pages that could theoretically be from the actual pre-32570 ledgers. Anyone has contacts to them or at least could crawl their pages to preserve what is there currently?
Sukrim
 
Posts: 1826
Joined: Mon May 20, 2013 10:44 am

Re: Pushing genesis back to 0

Postby JoelKatz » Fri May 23, 2014 12:26 am

The index of the metadata leaf nodes will equal the transaction ID. In prefix format (the format used in the node database), the index is stored as the last 256 bits. The hash prefix will be HashPrefix::txNode (SND\0) and the last 256 bits will be the transaction ID. So someone needs to go through all the hashnode databases and identify all entries whose first 4 bytes are (SND\0). For those nodes, they need to convert the last 256 bits to hex. Then we need to see if any of those hex strings are the IDs of transactions known to have occurred in the live ledger.

Note that the 'Hash' of these nodes is entirely irrelevant. It won't be anything special because it will depend on the details of the metadata. I believe the 'ObjType' will be 'N'.
User avatar
JoelKatz
Ripple
Ripple
 
Posts: 1859
Joined: Sun Dec 23, 2012 3:45 pm
Location: Oakland, CA

Re: Pushing genesis back to 0

Postby donch » Fri May 23, 2014 12:33 am

What you describe is exactly what I did in the earlier posts.

Edit: I calculated the Transaction Ids manually as a check. If you want to just extract the ones embedded in the node value:

Code: Select all
select substr(hex(Object),-64) from CommittedObjects where hex(Object) like '534E4400%';
Last edited by donch on Fri May 23, 2014 12:47 am, edited 1 time in total.
donch
 
Posts: 796
Joined: Mon Nov 18, 2013 8:07 pm

Re: Pushing genesis back to 0

Postby JoelKatz » Fri May 23, 2014 12:47 am

donch wrote:What you describe is exactly what I did in the earlier posts.
Then, unfortunately, it sounds like you're doing it right. If the transactions aren't in the transaction.db files and the metadata isn't in the hashnode files, then I can't think of where else they would be.

I am extremely puzzled though. There's absolutely no reason I can think of why they wouldn't be there.
User avatar
JoelKatz
Ripple
Ripple
 
Posts: 1859
Joined: Sun Dec 23, 2012 3:45 pm
Location: Oakland, CA

Re: Pushing genesis back to 0

Postby donch » Fri May 23, 2014 12:51 am

JoelKatz wrote:
donch wrote:What you describe is exactly what I did in the earlier posts.
Then, unfortunately, it sounds like you're doing it right. If the transactions aren't in the transaction.db files and the metadata isn't in the hashnode files, then I can't think of where else they would be.

I am extremely puzzled though. There's absolutely no reason I can think of why they wouldn't be there.


What was the history of those instances? ec1 looks like it was a separate testnet. The others look like they were working together intermittently, some live on EC2 and some local?
donch
 
Posts: 796
Joined: Mon Nov 18, 2013 8:07 pm

Re: Pushing genesis back to 0

Postby Sukrim » Fri May 23, 2014 1:02 am

I can confirm that none of the known transaction IDs are to be found - I did not even go the fancy route, I simply did this in python (comparing the last 256 bits of every blob to all known transaction ids):
Code: Select all
import sqlite3
conn = sqlite3.connect('hashnodedb1.db')
#conn = sqlite3.connect('hashnodedb2.db')
#conn = sqlite3.connect('hashnodedb3.db')
#conn = sqlite3.connect('hashnodeec1.db')
#conn = sqlite3.connect('hashnodeec2.db')
#conn = sqlite3.connect('hashnodeec3.db')
c = conn.cursor()

hashlist = [
# long list of known transaction IDs
]

for row in c.execute("SELECT * FROM CommittedObjects"):
    if str(row[3][-32:]).encode('hex') in hashlist:
        print "YAY, WE FOUND " + str(row[3][-32:]).encode('hex')


Quick'n'dirty for sure, but it takes a few seconds anyways at most to run through. :/

Most likely these instances did not run on the now current ledger - or at least did not store any data from that time.
Sukrim
 
Posts: 1826
Joined: Mon May 20, 2013 10:44 am

Re: Pushing genesis back to 0

Postby JoelKatz » Fri May 23, 2014 1:10 am

donch wrote:What was the history of those instances? ec1 looks like it was a separate testnet. The others look like they were working together intermittently, some live on EC2 and some local?
In the early days, when we were resetting the ledger, databases would sometimes be moved aside and sometimes not. There were three main live servers at the time and all were validating. When the loss of historical data occurred, we preserved the databases from all three servers, including some older ones.
User avatar
JoelKatz
Ripple
Ripple
 
Posts: 1859
Joined: Sun Dec 23, 2012 3:45 pm
Location: Oakland, CA

Re: Pushing genesis back to 0

Postby lucky331 » Fri May 23, 2014 1:24 am

i wish this gets resolved soon. i do not want chris larsen on stage being asked questions about this. this will raise doubts and some might even tag ripple as "broken" and "flawed". i wish there was something i could do to help.

what can the community do? let's all pitch in.
User avatar
lucky331
 
Posts: 2729
Joined: Sun Jun 02, 2013 11:22 am
Location: got jed?

Re: Pushing genesis back to 0

Postby donch » Fri May 23, 2014 1:28 am

JoelKatz wrote:
donch wrote:What was the history of those instances? ec1 looks like it was a separate testnet. The others look like they were working together intermittently, some live on EC2 and some local?
In the early days, when we were resetting the ledger, databases would sometimes be moved aside and sometimes not. There were three main live servers at the time and all were validating. When the loss of historical data occurred, we preserved the databases from all three servers, including some older ones.


This is an interesting report which shows the correlation between the dbs:

Code: Select all
ATTACH DATABASE 'ec1/hashnode.db' AS ec1;
ATTACH DATABASE 'ec2/hashnode.db' AS ec2;
ATTACH DATABASE 'ec3/hashnode.db' AS ec3;
ATTACH DATABASE 'db_old.1/hashnode.db' AS db1;
ATTACH DATABASE 'db_old.2/hashnode.db' AS db2;
ATTACH DATABASE 'db.3/hashnode.db' AS db3;

CREATE TEMP VIEW CommittedObjects AS
SELECT 'ec1' AS Source,* FROM ec1.CommittedObjects
UNION ALL
SELECT 'ec2' AS Source,* FROM ec2.CommittedObjects
UNION ALL
SELECT 'ec3' AS Source,* FROM ec3.CommittedObjects
UNION ALL
SELECT 'db1' AS Source,* FROM db1.CommittedObjects
UNION ALL
SELECT 'db2' AS Source,* FROM db2.CommittedObjects
UNION ALL
SELECT 'db3' AS Source,* FROM db3.CommittedObjects;

SELECT SUBSTR(HEX(Object),-64),GROUP_CONCAT(Source),GROUP_CONCAT(DISTINCT(LedgerIndex))
FROM CommittedObjects
WHERE HEX(Object) LIKE '534E4400%'
GROUP BY SUBSTR(HEX(Object),-64);


Code: Select all
0005E4B26D5F074319F1C4FB1EE481316CFD29A021148811F0BA2D81BF3A5E22|ec1|24744
01A445243228B8FF850601AC3430F5C5C5D33151094D93174300373D827626BF|ec1|23033
01B8A1AE0144384E262E694AB6E87598B7F01082502E2B41789CCBD8CCC1C3A4|ec1|23011
021A01FC7B769216E2D8C00B60A753CCE6C6FAE8427015B9BFDC525995EC9C11|ec2,ec3,db3|3182
034C124C96792BFA7735E8C5CE6644CB95A9BDE87718B21CF7D39945328BB04E|ec1|22844
0378E7F54028350D3A393A9AA05080CD70FC9BDB227FE77BAB17EE90A0B25693|db1,db2|93781
039331252C5E61DB1468EA776720555D0A9186B119AE6FDCDBBEBEFE58C131A2|ec1|38142
03A949548B185F7194BA5567E94BF0D4417985CE7A4809B4FF217B70D0C4B4CD|ec2,ec3,db3|5164
055B03692D3569EAF4D399975D0EFC099E198BAF8B94AE3A1358A8AAAB12AD7D|ec2,ec3,db3|9146
05A50D16D27717D3010DFE5CA88C1455D0653921CE53C9F63F23414A57607C10|ec1|36772
05C2A089B0B0E905F5920787703536B4173BDCD57D457827EC86A42323F73F59|db1,db2|114298
05CC92959F1CBFCCA419DC247C6B6FDC0B7CB75B20B41CA61AE44DEAF3DB1B44|ec1|33608
063F0BE61E8752A376F836B79CAF8DF956C6351DDA64C1A02D1BB5A88ED3E545|db1,db2|36419
068B290AFFB16D0468CA2BBF0B0689E052F6B0F482140AA1D9559173B5915B16|ec1|62215
071F4F846B8CC1D35F698F619DF9E24E9D619DD0850115AB3E78382EACFC9F6E|ec2,ec3,db3|8420
07DDBF3438E2C28AEB5D7DFF5DDDD433A1429211E68BE56793EA1471BB58E7B3|ec1|36590
08222B1B037D43CEF0D9D939C518F313E5A7D9FCB63FE5497ACDF87CD899358C|ec1|44559
0913BC8F11CE23147DFE9DE4E15E49C5224A9F4BC62A62916776B08D9CBEB2D8|ec1|62222
09FA8E264CC0EF57D9B7E45CE6ECE3B5E9396B55D950B884EDF52CD76F34F5AB|ec1|18678
0A01CDBEDDEEE815D1FCBB71740ECEAA9EE6211A9498EBC2ACE44A729C647EF4|db1,db2|114573
0A5F8423E3708E6031471E6100221A88909F6FE7513093DEF9C414588E8F714A|ec1|34389
0AA6818964E882DA23D2335788B0B51B990163F7C7F1B14F55AE9AAD20541364|ec1|70584
0B14A856E379F7941213CE4633EFCF1BBF87C400C44B43E3157882DAE7EE779D|ec1|57800
0B45043DAF35C738CC39F9B539FC817F869E021127F214BA1FB88C526AC2386E|ec1|25500
0B8E5A6AD47C35560340B01EA061021BA5CE53667CC447A928C974AD01F37B1C|ec2,ec3,db3|8722
0BD044C8C1FC2E8E357FF657A5BA1ED82DB1073AE53F492CA3947D99C6389B6C|ec1|33390
0C1625440D10941BD05677B4594D05EC7E442BD9E26C304A5575904697E71FCE|ec2,ec3,db3|3333
0CD16C5B0E50C1B715D60EB73353A7C08A85A53F868B16A95347FA32EA7BD663|db1,db2|60776
0D488243FCC925C58705D276034C2B05BE6CBCF7710906572E6326F0E2F7BF02|ec2,ec3,db3|3097
0D5F7795971D048A5743ED15486B62178658DA8CE0B07D829DF2180CFC324DEA|ec1|28441
0D728B94980CB93346872E650753D2A61994C620724DF48292F3A9CA753558CA|ec1|7419
0D7D060529AB8819AB7AAA0F5A3FB0F248FC88A58EEDF5D09EAB9125F2B805E2|ec2,ec3,db3|7767
0DA5B165BB9EA8DE1711A8FDE8E054B24C2E09040A5D1C10360BD9F409D1BE36|ec2,ec3,db3|8558
0DEB9D2A7E2457E2718934496CF0F0224011F593D5FE81586F905F59084ADC5A|ec2,ec3,db3|13390
0EA095458F644355873570C4B49BE3A9281F4A621A30F3B8349A45F76AE83A40|ec1|23048
0EA85371A766160551EBD7477ABFD1D2EF2755576C0A3F5272A6ABDB52FF6014|ec1|25082
0F312C5309BA48FA3C985A99201B7E89318B65CF76FDF5035CC87CA1316868E1|ec1|30135
0F6B0E54E04F58FC97985373F2E446445822627A4DFD16842903784082790848|db1,db2|46872
0F824C6057738FA58841B1241227545AB3715D41560AF3DCE6968740C1E4D66C|ec1|42494
0FE8F2BDCC3628A1BF1115899C66A1E11381DCD80DB7E1DBAB81604A880F57B9|ec2,ec3,db3|5160
0FF0A11161E28F2A157E0FEFE73431A21835AF78400A4C690D51768CA2882DA3|ec1|43635
102F480A4BEEE2C2F994ADA387DF1342C5D241400A40950B83F2D55BC5491A98|ec1|24887
104618E69B5990F71C1570230CCA50616D0FE03AB097E7E9FD571A52BFC0EDC4|ec1|28527
10AC0A1241805316F74521CA28E9B40383E3F0FD50CA564B78E82A8ED94FD081|ec2,ec3,db3|8680
10DBCE498CC21070D4932F294912341C3974083B812D3B7AD2D95EB041423EAE|db1,db2|93839
10EEC2A4F196CC0D7CCFC172272302CF062D5CC2083BB3A538B05C2942D5D1A6|db1,db2|57947
111219DE7B392A01F4D107A0355A9015CEF31F3F71FFBC2C453365E1051B0E71|ec1|32503
111AFEC2E0D8DA3720CBC30CA477ADDA2CF20C3E85605BDCB3B50C1BC03126C6|ec2,ec3,db3|3272
119C672442156AA206712D54422AF9BCAEE592DE359F1C92990F066A68F905C7|ec2,ec3,db3|4859
119D626821E8B498BC6E52D6F969E4A77186EF3415E07FA7070D79FA8138505F|ec2,ec3,db3|9585
11BCBC7DE381D402BCFDDB6DBDF4DBAA8AE98B8A229AE004B11936F18ABAB948|ec1|30131
1226137D4D780A65B04D14694CD59CCFDFA4C8FD567BE861227600B2D4CEF235|ec1|34268
1237277FEB5A53B7F8DF590CF07B6F7AB42805433997D80EA2B09EAD283646C6|ec2,ec3,db3|3155
13223C02044D397ACB74F92C40B8C621C5439B2AAC7EDAA9EE28055F599D8A63|ec2,ec3,db3|5266
1381296D5F5A5262E3AFC28FDEF5D7744DF56061A3AAB9AA16A14415EC495EDB|ec2,ec3,db3|3337
13853D1B8C73CAB325202E945C992CB8EA152B0A22E3060E7DE3315ACCF6AA01|ec1|29424
13C33D9151033384AB17CD4D8699F1DE8606BDAA7A299D7A080E28E0A2079C67|ec1|19275
13F897858226610C94B2DDB3D0AFCB3196562DD918E56A449769944A75D6BDFE|ec1|36647
146FC93AC2EBC25CB5D2350E2EEF3A5177CD06C1268D1496E5FB99ADC92DDDB2|db1,db2|60820
149994D4AEF3CC6188F9DE728CD1F601A8CDEDFBE3544FE00E2EB055AFC8B33F|ec1|36597
14ADCD3825024FCFFFC1713358C0B79BEA4586A0C66F1CEBC573661E916E3182|ec1|28500
1524B0E42C22DDB5A8AF19DF6D2629382BE1986E6C46018262A862C9D4D7E6D7|db1,db2|56232
154856BC57CFAFE5A35E18CAE4A7EBEA5364B1D9AF0AFB46AAAD46DE6AF4EAD3|ec2,ec3,db3|13399
15DCAF857FD944765A0C0D2912E4ED22EBB7740626C6B039238867AAAF6F6C84|ec1|25642
16072D19294BF612A99F6D315DAB2F6C7354452F1E3B843771FFBF6363603521|ec1|23811
161EA37D35E78D4B52559EE0657A3B9F19A084178199B7F596AE501B55E82077|ec1|33055
16EECE4D18077FC2EFAB1B1DBDA1979C893FFAC942708916C0541BC91E60FB39|ec1|51951
170750EF4289E1EDF9F83896141DA4EFED860551B6A33BBB465ABF06945E38AA|ec1|36592
1724A56930540D89A0749B8E063D892F0405278A85CB7ADAD1A258C3DD3573BF|ec2,ec3,db3|5597
17C538FAC4955F8051DB444A0D06E56F8A3B719AB174E3F6B9046896716BE695|ec1|38378
17D92AC6A4B722B6AF63A89F5BC9E96FFF9D181F4710DD1ACEDF5A76F8389835|ec1|22995
17F29C9741FBE265C903EE34B5D504BA3E799918A8A7C740B4119057DA06B52C|ec1|32557
1826CBE368A0B4B56FCAC7C2D09347892ECBA8B66DE2DAE5E9A55D7AD5A48664|ec1|23221
185974AA208AB2E11640494828A3F9FE89D3FDAC8A3566B4B4D12B0814F03DA8|ec1|23036
19230BCF8D679A2996E859C69C1DAD4B3DAE9FC4F41B865F846BF0F812CC5A12|ec1|33088
199FBF3C503B0ABF0AEE2767A38DCDD4A4EC54A0EE6F90403496BA60988F53C3|ec2,ec3,db3|8804
19E4E6E0223FEA111E6F8532CFAF9A15622961CF0432ED61CC9083619BD60553|ec2,ec3,db3|917
1A3DC1E0D296FF6FE81E552D8DF9C59D91E7D39636B7708A24AFA6A85CDD6F19|ec1|42896
1A4B208EAA188493042B593925415459D13967318BBCC38A043C438D18120C68|ec1|33595
1A7C1B026CC5E10E70737CE89C767C8DF00788E2A3284E197A848764C55773FA|ec1|30144
1AAB2D061D16236A7977A561ABC0F21183A9E06E51AB6D943794D6083C3EC7D2|ec1|62010
1B197DB77523B38F7A89663A2D995CD3AF5837B6FD5FEE1C3EE39E7DBA58671B|ec1|48658
1B49FF7F371933EA259383182E9FEBFAF6EB36390381194E89B1990D2734D479|ec1|32505
1B683E53342DC2CE206A57CCDD7345F62352FD64FDD5D185FCA100002A260000|db1|10818
1B953D95562F7891C40FFFCEC9DD0B52A7DCDB1D7E0B7249E83A6D1C4CD6C317|ec1|28268
1B97C8E00FA415169ADA97A9C5E6B5D2053F984E1E1369CC38BA06DFBF706CAE|ec1|51831
1C31463E86F561B2B5DC0F00C4DA9B424CD6213437EC5E15B1F92FD7A39E37DF|ec1|18668
1C6BA404F624F654456250D4A490B6AD941EC9CAAADD64745CC96636AC4AD308|ec1|28489
1C870D5FD185FAF3212D93456BD75B5827874E943E356AA890D20D3B81FEE4BD|ec2,ec3,db3|7780
1CBDA1A9549EAF956E31DBA8DE5C8608F838032D3DD5681D784DFF2A6E9ECACA|ec1|18764
1CC0B520A55004FC3735BFDE572B3CBF2B52F5FE48034FD30C76C03B3CC6FFD4|db1,db2|60781
1DAAC92D1275C959E6BEB33764B3D34F9714CF1B553B4DA4221779EAA23C9E49|ec2,ec3,db3|3199
1E32504D3D522EF9E1EBF1AEFA384462143A6C9D640CBB1AFF063727E6A95C12|ec1|38067
1E7876B252A21145465F149F22C84EA2818C0C05C679B022CC87F1BC2D1B24CD|ec1|38136
1E850810D6BDECEFD9BF00F293FF6D4476AF7F45B82A1062AE7C746815A93E57|ec1|32644
1F47B5D8F7746F83B88C7D6DA848F774D89AF9DE35A9CB980C43375A947525BF|ec1|7403
1FBBC87F4A6924EBF2526A84024E60F3A775B6715E201D7ED4471DE166FA1425|db1,db2|22517
209BDD76CBE424226023727156349226BE1E021907E6D10BBDBBB128702699EC|ec1|23808
20BD0F50ECBFB90074F03B85BBEA6CE99EACA726F6D0CA8BFC35CD828ED7BC73|ec1|71196
20D9569AFE363A5CA460BA29CF1F49F6F825682AEFD33B2F536AFB94E40DD32A|db1,db2|57982
20E72C7B2BB611A7D8B949C11972A90ADA57139D423A178576C5FD5DB9D5A7CB|ec1|42642
214AE8A2CDF1949CAAAA92BCAF104370A9D880AAA51A97C160B7266EB7BABFE6|db1,db2|26322
217544B78B6B98D911D8298E0CDB31CDF6920C932D9553C6C9577B2643462688|ec2,ec3,db3|5606
21BE54248954C0CC2A034B73FC2CC30B0C8A604466277583FB4C3251ACFDCB73|ec1|46558
221E0FBEB016F30E5B2945E8883C278E9E96E768DC53D23679006503E05BBE41|ec1|57556
22682015CC2788AAD84471F18680F6050BAA850A1842DBF582C51ED26CBBB9A6|ec1|36773
22C7DF2A8E32BAAA286F9CA67D60A25AD4EEFBEAAD5FFB56A6FE9B5D900FF207|ec1|29832
2339BF170989D22933F65BF1DBC97A648CFB642E9E678EEADCBE4778096F2338|ec2,ec3,db3|8228
2371A60BC9D3239AA15EF361044C3E5E3B245660523976CFE221AEBB30B897B1|ec1|38082
23956BFF5D3CE339B46936C57C9928B4267901130C08E06AEA369201D4EBE09C|ec1|25021
240E1FED85FBF6C25AF78E45286C0B7CA10AE71B98827F363D5E21930E195E2A|ec1|25670
2487A976A0A2EF1BDA307BF5FE55C3E2FDA083F8A4524D3AAEFB02CD7A7AD481|ec1|36616
24C57FA0C6D478CA9BDA35E0B3D355C654D72FC213569AF81DCEC3690153BE39|ec1|5263
2513AAD8D758A65CC2C91F07D8B895F2A93509B15D3893CD49D2E376921AED85|ec1|28523
2518A266D3800B650B0642FC46C1073BC343186E1E21EC9651F2E949873DEF15|ec1|30052
251FC96BA369D47211589A76273BFB9E1BE7D9FF74489B6A8DAED535B6529455|ec1|28438
2543573F0FE6B19091EA197D61F97A7B1EBEF5175C7728E0593CBB61AC5915FF|ec1|469
256F6DF30D0D1FC9407BB78305E13DC0D91D0816A6312AA5C4370A290C64DC15|db1,db2|36529
25994F5F017F75A5F247E0C3094C9C57E164C298CBC71E1573BAB4E7CD304674|ec1|12124
25B4F3919F1B85FF54447BF3EB029A2D24986F4817168740A02B5E6C3C7CDCA3|ec1|70631
25E2576C3B89573B438C5024C79FF1F16AB5F0D8231B688D65D151C7CB5D583F|ec1|38132
260C4A204BFF8BBAE86229EF2263EB4EE3641E1EB692EE9EF411618AD4C32F86|ec1|18681
261E8FEEF2545168FD26160BBE440D821A20C4A5D8C7146094821B7F8E6DC39D|ec1|33692
266D41D1225870201C7357692B57234365D996CF08601A8DEE213001B52A61FC|ec1|34236
270418EA55B1B25CFA4ABD522166E42C95D5C9286FC78CA56A301E4A10DD60A7|db1,db2|31487
2709866A4044DB9107CB5DCA3943E124B147F6D2A8679EC4474E3BA66B6568C6|ec2,ec3,db3|3229
284500D8DC8B5C1650F0CE6F61AE92166624076EBCB1BC8E75233D4C937D8FCE|ec2,ec3,db3|9773
2850E7952B73C01221BD5D96AAB679DE7DE7BF3608A6FBA3B8EAC9AE8ABB1778|ec2,ec3,db3|746
2928C6551992A39CD2E9BDAA5819A3E06EF853DCEE3C8432342E044457020DB0|ec1|49444
29841546CC68F1EB0A5C6BAA64E4485A93138BB7FB6A374EA263B9CC1C2EF200|ec1|42208
...
...
// too big for phpbb!!!
donch
 
Posts: 796
Joined: Mon Nov 18, 2013 8:07 pm

PreviousNext

Return to Developers

Who is online

Users browsing this forum: No registered users and 7 guests
cron