Corrupted pastebin upload.

[Dr Super Good]

Drag and dropped a jpg image into chat to automatically upload to pastebin and link it and the image uploaded corrupt. I have attached the original and resulting image to this post to demonstrate the corruption that occurred during the upload.

While uploading the upload stalled for a while then resumed. This is likely when the file corruption occurred. Instead of continuing to upload garbage data it would be better if the upload failed and the user is instructed to try again. Re-uploading worked fine, however not everyone may notice immediately if one of their uploads gets corrupted.

If this happened to a map or model then the map would not be openable and may cause crashes.

Original

Corrupted

Corruption can be seen at the lower half of the image.

 

[Ralle]

Wow that's weird. Does it happen every single time you do it?

 

[Dr Super Good]

Does it happen every single time you do it?

No, its the first time it has ever happened.

The upload progress bar in the chatroom froze for several seconds before resuming. One can assume that some sort of connection problem occurred. During/after that time the file must have been damaged.

Rather than accepting a damaged file which is probably useless, an error should be thrown asking the user to try again. Knowing a file upload has failed gives the user a chance to immediately try and upload it again. If the file is damaged it may only be noticed some time after being downloaded and fixing it may not be convenient to the uploader at that time.

Just to make it perfectly clear, the Original image does look correct. It may look like it is showing something messed up but that is its point as it is trying to show off a test texture I made. The file corruption/damage being referred to by this problem is the difference added to the image that is present in the Corrupted image.

Difference

 

[Dr Super Good]

Sorry for double post but I just had this happen again with a map uploaded directly to the paste bin (not through chat). Something is seriously wrong with the error checking as it gives no warning that some error occured.

As I feared, I did not spot this and now the guy is wondering why it will not open...

My upload rate is over 1 MB/sec. Maybe this only occurs at higher upload speeds.

 

[Ralle]

That's really weird. The error checking is part of the TCP stack. Technically, errors can occur but most of the time they are caught. There's not really something I can do.

If we make enough experiments perhaps we can complain to our server host that packages get corrupted.

 

[Dr Super Good]

A good place to start looking would be what sort of damage is being done to the files. That might explain where it is being introduced. I agree the TCP connection itself should not be the source due to inbuilt error checking, however that does not stop whatever is using the TCP connection from introducing the damage due to some logical errors.

For example, what happens if a TCP connection closes on one side? Would it attempt to resume the upload by opening a new one? If so how does it know from where it must resume sending the data?

I am not familiar with out websites handle file uploads, but in theory one could detect such damage using cryptographic hashes. When a client orders a file upload the client's browser takes a MD5 or SHA hash of the file and then uploads the file followed afterwards by the hash. After the upload finishes the server re-computes the hash and compares it with the one the client sent. If a mismatch occurred the file was damaged and so can be deleted with the client notified to try again.

 

[Ralle]

Nobody in the world actually do that for websites though.