TTimoNN

http://rtor.nyx.feralhosting.com/files/WoT_DLC.torrent

It's been sitting there for a while but I can't get stable performance out of it. Neither can I finish it 100% before it either disconnects or comes up with a hash error.

If you have any issues with the DLC server please post them in this thread.

Please provide as much information as you can. (traceroute, speedtest)

For 404 errors please contact Aslain directly.

Current status:

2020-06-20: We have moved to a new hosting provider located in Falkenstein, Germany. Much more stable solution but long-distance routes seem to have less throughput.

There are no plans to use a global CDN. But we might add individual servers in locations with a really bad global connection.

old:

2020-09-29: small issue caused the DLC server to be offline for 2-3 hours. one-off event that we can easily prevent in the future.
2020-06-18: ISP seems to have locked up the webserver without communication.... currently investigating.
2020-06-19: Throttled all individual connections to 2MB/s, will look into more stable alternative solutions after this incident.

2020-05: IO issues currently, standby.

Above has been fixed. Moved to a better storage solution.

Slow download? first check if it's your own connection with a speedtest.

Some remote locations can have very slow download speeds due to the DLC server being centralised in Western Europe, feel free to post your country, speedtest result & download speed of the Aslain modpack if you have this issue.

20 hours ago, SoftwareSimian said:

But in a torrent the client will resume the transfer and check for corruption

Not when leeching from a web seed sadly.

The repo has HTTPS support so you can always use that.

I'm still actively working on the issue by trying to fix it through other means.

Web seed over torrent caused the same issues that regular http caused. Will look for other methods of resolving the problem.

If anyone here wants to actively debug this and lives (or has a server) in either America or Asia (or any other location that seems to have download interuptions) contact me in pm on this forum.

10 minutes ago, SoftwareSimian said:

or you could automate it using something like mktorrent (a simple cron script could check for any files in directory modified since .torrent was generated and regenerate it).

Sounds like a plan, I'll use web-seed to bypass having to write another script to add and remove it from an actual torrent client.

We'll add it as a link either in the installer or somewhere around the download location. I'll respond to this thread once this is up and running.

Thanks for the suggestion.

9 minutes ago, SoftwareSimian said:

You can see it's throttled to 170kB/s and then loses connection every 3 minutes or so. wget can resume the transfer, the installer does not.

Looks like somehow your local connection doesn't like big HTTP transfers, seeing as you also have trouble downloading them from your own (US-based(I think)) server. I have no idea how or why this is happening... maybe your ISP is throttling something. (wouldn't image so but you never know)

15 minutes ago, SoftwareSimian said:

It actually started about 2MB/s and ramped up to about 9MB/s before it finished, more than the average would indicate. So the source server is not the problem.

9MB/s is still not by far reaching the potential speed that server-to-server would give us. Is your server connection 1Gb/s or 100Mb/s?

21 minutes ago, SoftwareSimian said:

So now I have a "local" copy on my server, I tried downloading from there. Slightly better, but not really: 314kB/s.

However, let me try the same download via FTP instead of HTTP: 4.8MB/s

So... setting up FTP is doable if the installer supports it. (waiting for a confirm from Aslain on that one)

Nothing wrong with giving people a choice between ftp/http ofcourse.

31 minutes ago, SoftwareSimian said:

1) is there any mechanism to allow the installer to resume DLC downloads if the connection is interrupted? Or, at the very least, can the failed-download be detected and warn the user?

see ^

32 minutes ago, SoftwareSimian said:

2) I would still love to see all the DLC put into a torrent both for faster download speed (for us poor unfortunate North Americans) but more importantly for download integrity assurance (since HTTP downloads have no checksum). I suppose now that I know the download location I could build that myself, but it would be nice to see something official

Right now there only is support for downloading the installer itself through the torrent platform.

We'd have to either make a torrent for every single file, or throw all files in a single torrent and only download part of that torrent.

Problem with the last solution is that these files get changed quite often. So a new torrent has to be generated every time a small update takes place.

It's possible but from my pov inefficient. Also the installer would have to support the torrent protocol.

I have no experience in programming installers so again, Aslain has to look into that.

As for the downloads being interrupted every few minutes over the HTTP protocol, do you also see this happening on your own server or only on the DLC server?

offtopic: thanks for the extended debugging on your part. wget is love wget is life.

19 hours ago, SoftwareSimian said:

1) Put DLC on a faster server

The server has a "shared" (<25% use on average):

2x 10Gbit/s exit to Switzerland

1x 10Gbit/s exit to US-California

2x 10Gbit/s exit to NL-Amsterdam

1x 10Gbit/s exit to NL-Utrecht

With a combined cap of 10Gbit/s out of which on average we only use 1-2Gbit/s.

The server/web server supports upto 100k concurrent connections. which we are nowhere close to.

This is not a speed issue.

19 hours ago, SoftwareSimian said:

2) Assuming that's not possible, please make the DLC available for separate download (torrent would be ideal, http would be acceptable). I suspect that it's not possible to have the installer actually download the DLC via torrent, but if the DLC could be linked for download somewhere that would let me (and anyone else with this problem) at least able grab it to put in Aslains_DLC_cache folder.

Raw files are located here: http://rtor.nyx.feralhosting.com/files/

5 hours ago, Davidinlv82 said:

Another thing to note too Aslain we have crappy internet here in north america compared to European countries, the infrastructure suck here and they are doing very little to make it better yet charge us an arm and leg for a ton of nothing but disappointment. Just wanted to let you know.... 

This is most likely the case. Indeed the server is located in Europe but there is at least 1 dedicated line running to the West Coast, I can't provide more than that with a 0$ budget. Chances are that your ISPs infrastructure is not optimized for high bandwidth cross-Atlantic traffic.

Try downloading some big files directly through HTTP and see if you can get some decent speeds on those.