IF Archive uploading info

General IF Community Resources
1

Someone asked “What does it mean that my game is in the unprocessed directory?” And I realized, hey, this isn’t written down anywhere.

So I figure it should be.

NOTE: I intend to update this head post as our procedures evolve. Feel free to comment or ask questions in this thread, but remember that the info at the top will change.

Uploading

There are two web forms that allow you to upload a file to the IF Archive. They’re very similar.

Both forms ask for your name, email address, a description of the file, and a suggested directory to place it in. (That last is optional; don’t worry if you don’t know.) They also ask whether you’re uploading your own work, and to agree to the IF Archive terms of use.

What’s the difference between the two upload forms? They both do the same thing. The IFDB form is just a convenience you can use while adding a game listing. If you use that form, your upload is associated with the IFDB entry, which makes it easier to update the IFDB entry when the file is moved to its final home on the Archive.

Once you complete the form and hit “Upload File”, the page should respond “Thank You For Your Upload”. If it doesn’t, something went wrong.

(It should explain what went wrong, but, you know, not if something went wrong with the explain-what-went-wrong page. Could happen.)

Incoming files

Uploaded files are placed in a private directory called “incoming”. You can’t see the incoming directory. This is on purpose. We have to filter out spam, off-topic trash, and “whoops that was the wrong file here’s the right one.”

So a (human) moderator does that, and then moves the file to the unprocessed directory. At this point everybody can see it. (As you can tell from the fact that I put a link there.)

Historically, it took a few days for files to move from incoming to unprocessed. This is because there was just one or two people doing it. We are now bringing more volunteers on board.

I think we want a target of moving files to unprocessed within 24 hours. We’re not there yet. We’ll see if it’s reasonable.

Unprocessed files

The next step is more work. The file has to be moved to its final home – games/twine, for example. Also, we have to write the little blurb that says “Game Title, by Person”. With maybe an IFDB and/or IFWiki link.

Historically, this takes… however long it takes. A volunteer logs in occasionally and moves a bunch of files. I know it doesn’t sound like a lot of work, but over time it adds up to a lot of work. People do it when they have the energy.

We’d like this stage to be faster, but I don’t know what the reasonable goal looks like.

Once the file is moved, we email the uploader to say that it’s at its final home. (I think! I’m not actually sure if David Kinder did that for all files. It seems like a nice goal though.)

If the file was uploaded via IFDB, the IFDB entry should be automatically updated to refer to the new location.

Updating your upload

You can always upload a new version of your game. Mention that it is a new version, and what the new version number is (or whatever). Please also fill in the “suggested directory” to refer to where the old version of the file lives.

We won’t throw away the old version. It’ll be moved to an “old” subdirectory (like games/twine/old) so that future historians can do their jobs.

12 Likes
2

Note that right now, the unprocessed directory has a big backlog. Worse than usual. Sorry about that.

This is a combination of a few factors. We’re in the middle of revamping our procedures and orienting new volunteers; a whole bunch of old Adrift minicomps arrived at once; and DavidK is out sick.

I will try to get some of this sorted away this weekend.

4 Likes
3

I remember seeing that directory alongside with unprocessed far in the past. Was it ever visible to regular users at ifarchive.org or present in ftp.gmd.de?

4

If I understand correctly, the old FTP-based archive used a “write-only” incoming directory for user uploads. (That is, you could see the folder and upload files into it, but it would always appear empty.)

2 Likes
5

I am ready to get to work on the archive. I would just like some direction so I don’t bork something.

1 Like
6

I’m figuring it out myself… we shall coordinate on the IFTF Slack.

3 Likes
7

Question on IFArchive naming of files.

My entry is archived as “One King to Loot them All.zip” and its comp update is currently in the unprocessed folder (I was told that IFArchive would be updated with the latest release uploaded during the comp but apparently I have to do that upload myself), but I noticed the “view contents” link gives an error for the Unboxing Service:

NotFoundError: Unknown file: /if-archive/unprocessed/One King to Loot them All.zip

Is this because the filename contains spaces? Should I have used underscores instead? I do not recall reading anywhere about recommended file naming of IFComp entries. The entry I want to replace with my upload can be viewed in the Unboxing Service:

https://unbox.ifarchive.org/?url=/if-archive/games/competition2023/Games/One%20King%20to%20Loot%20them%20All.zip

2 Likes
8

Historically, the IFArchive has tended to use underscores rather than spaces, but that really is just an artifact of its origins in the ancient past as an FTP site, where spaces would be an annoyance at times. There’s no good reason for not using spaces and indeed, it is supposed to all work for files with spaces in.

The unboxing service should handle this, so I have to admit that I don’t know why it doesn’t here in unprocessed, while it does for the version in the competition directory. @Dannii do you have any suggestions?

1 Like
9

The unboxer checks for updates every 6 hours. Problem is if you look for a file it doesn’t know about, then the error page will be cached for a week. We can reduce that time but I haven’t done it yet.

I had set 6 hours cause the index used to be updated only occasionally. With more people involved in processing uploads maybe it should be dropped down to only a few minutes. And I think error pages should be cached for only 1 day.

3 Likes
10

Could this information be added to the form itself, along with instructions on what people should do if they don’t get the “Thank you” message?

3 Likes
11

It does a HEAD check before downloading Master-Index, right? Dropping that down to a few minutes is fine.

And I think error pages should be cached for only 1 day.

Reasonable.

12

I don’t want to fill up the form with info on everything that can go wrong, because then people will think it always goes wrong. (We already get questions like “It says it uploaded successfully, but is it lying?”)

I will add a link to this doc post.

3 Likes
13

You could phrase it in a positive way, like

Once you complete the form and hit “Upload File”, the page will respond “Thank You For Your Upload.” That means it has uploaded successfully.

But obviously do what you think is best.

2 Likes
14

Giving people some kind of time frame is helpful.

The files will be processed within 2 weeks. (Or later as the schedule permits)

This needs to be somewhat accurate, though, so people who check things out should look at the workload and projects how long it will take to process the jobs, and update accordingly.

15

It certainly would be. I look forward to finding out what kind of time frame we can support.

2 Likes
16

I absolutely did send email for every upload (except for the automated backup uploads). Pretty sure David does this too.

1 Like
17

Yes, I would send an email if there was a valid email address in the upload form. Possibly only one for the final upload if someone uploaded a bunch of stuff, and I might not bother for very long-term contributors who know what’s going on (like Zarf uploading a new version of GlkOte, or something). For most game uploads I’d try to provide a link to the index page, the direct download and, if the file was zipped, an unboxing link for on-line play.

2 Likes
18

Mr. Plotkin, is it possible to upload files anonymously, or is this forbidden?

19

Hello, Onno.

You’re correct, I did likely tell you that, because we used to do so. During this competition, I learned that for the past few years we have not continued to do this, but I was unaware. Apologies for the misinformation.

We are looking at (potentially) automating the zip file creation process to lighten the amount of work it takes to create the zip and, if that happens, going back to creating a second zip at the end of the competition.

Thanks (and again, sorry for the bad info!),

  • Jacqueline
3 Likes
20

You can do it – the form doesn’t require you to fill in a name or email address.

However, in those cases, we have no way of checking where the file came from, whether it’s authentic, or whether it was uploaded by someone with permission to decide what happens to it. So we would look at it with more suspicion when deciding whether it was spam.

1 Like