Community Wad - A new direction?

[Xander9009]

For the contributors who are active, I've got some news and would like your opinions.

My dad and I have gotten a server up and running with a website I can access and modify as needed. I'll be attempting to get this website to do what I need for the distribution of the decks. By that I mean a site with a list of the decks and what they do which can be programatically updated with the actual deck files as they come in rather than the Google Sites one that currently exists and requires me to manually update it (which is why it's so behind all the time). It'll also allow for including when a deck was made and thus sorting and filtering for new ones.

Now, THAT is useful for everyone. What's not useful for everyone and would instead only be useful for the contributors is the ftp service I've set up alongside it. I've tested it and it works pretty well. I've had no problems with it so far.

As it stands, using Notepad++, here's what it takes to modify a card:

1. Open Notepad++
2. Click connect (the plugin stays open if you had it open when you closed N++)
3. Find and open the card you want to modify in the list (you can scroll to it, or type its name to jump around in the list; open by enter or double clicking).
4. Modify it as needed.
5. Save

Adding a file is even easier.

1. Have the file somewhere already (probably your testing folder) and the server connected (click connect).
2. Click the upload button.

It's uploaded with its current name to the current folder. Each user will have access to a folder containing the CARDS, FUNCTIONS, and TEXT_PERMANENTS folders. However, in N++ you can set it to start off in CARDS by default. So, uploading a file would always put it directly into that folder unless you change to the others.

In either case, once it's uploaded, the repacking utility can take over like normal. All without the need for Google Drive, an internet browser (because I HATE Google Drive's webpage), or potential duplicates of files (one of Google Drive's main problems).

Each user would have his or her own username and password. Unfortunately, I cannot allow you to make your own passwords without me seeing them. I want to, but it's not a feature of the FTP server I have (FileZilla) and I can't afford a paid one which might have that feature. So, you can make it whatever you want (and I will not be storing those passwords anywhere), but even so, I'd make it something you don't intend to use elsewhere (or at least anywhere else that's important). That would of course all be handled through PMs.

Is this something the contributors would be interested in trying?

For the sake of making sure everyone keeps decent download speeds, the wads themselves and the images will remain on Google Drive. They're not modified often enough to justify putting on a slow ftp server. For text files, however, it's almost instantaneous. It takes a moment, but it's less than a second (although I am in the same town as the server).

If you use another text editor and want to continue using it, you can. I didn't realize Notepad++ came with a bundled FTP plugin. Yours may as well. Sublime Text 2 is not packages with an FTP plugin. However, I've tested Sublime SFTP and it seems to work. I don't quite like it as much as N++'s, but it works well.

You can also use any editor you already use and simply upload with whatever ftp client you want. FileZilla's own client is really nice. In fact, from directly in its client, you can right click the file you want to edit and choose to view/edit and it will immediately download and open in your default editor. When you go back to the client's window, if the file has been modified, it will ask if you want to upload the new copy (with a checkbox asking if you want to also remove the temporary local copy). I find this to be a slightly better way to use Sublime. Use the FileZilla client to open the files, with sublime only handling what it's best at: text editing. The only real downside I've seen is that it doesn't allow for batch confirmation. If you edit three files, it'll ask three times what to do with the modified files.

So who thinks this would be a better way to go? I certainly do, but I'm not the only one that uses it. Only one method will be supported. They'll both be available for a time, but FTP or Google Drive will be used and the other not after awhile (probably; it would simply be twice as complicated as it needs to be if I have two loose CARDS folders with 10000+ files each). Google Drive has proven to have quite a few flaws where this project is concerned, and the FTP method should be quite a bit simpler once you're set up. It also means you don't have to have google drive running. I can get the download links for the wads and make them all available (possibly giving a download manager the ability to handle them all simultaneously), and the versions can be checked through a file on the website, allowing the syncing utility to check for and download the files when you want it to rather than immediately or not at all. This would mean users also wouldn't need the Google Drive program on their computer.

One other benefit is that anyone can view the list of cards and see things like the code and when it was changed without having to wait for Google Drive's website to load. (If you don't have the loose files in your drive, then finding a particular card is a nightmare.)

I'm interested in your input for this. I really think it would be a better way to go, but I can't see into the future and I don't know for sure how you use the system currently. So, it's possible you'll find it as easy and convenient as I do, or it may not work at all for you. Let me know one way or the other.

[Armodeus]

Absolutely! FTP is what came to my mind the first time I opened that garbage from google.

[Xander9009]

I've just confirmed that if you try to upload a file with the exact same filename as one already extant, it will overwrite it rather than creating a duplicate by appending (1). (Before, I was assuming they would be edited directly from the FTP site to avoid this, but it seems it doesn't need to be avoided because it simply doesn't happen with this setup.)

If we do it this way, I may split the cards up into multiple folders. They would just be sorted into a folder based on the first letter of their filename to avoid having 10000+ files in a single folder. Once I had all of the cards in the folder, it still loaded fairly quickly, but there was noticeable lag. So, with ~10 different people, it may be a bigger hit. I'll see how that goes.

EDIT: Conclusion: it loads a LOT faster when it first just has to load a list of 27 folders. I'll let you guys try it out both ways and see which way you'd prefer it set up. It won't matter to me a whole lot. All in one folder is slightly more convenient since you don't have to choose a folder before choosing a card and you can upload without having to choose a folder first (I'll probably make the repacker move them when it repacks, so that would still be possible). With them in multiple folders, they feel less like an endless wall of cards and more like a structured collection and they load faster.

Also, something I just discovered: FileZilla's FTP client has a filtering feature to allow you to prevent certain files from being shown. Specifically, you can prevent Window's desktop.ini files from showing up, and you can easily set it to filter web.config files (they are created by the website software to manage the folders).

[nachonal986]

Hey, I have experience to work into FTP for share custom works of games, in addiction I had to pay U$20.00 for have an account and download stuff of an engine mods called HyperSpin, well nevermind.

The FTP is a good solution, and I completely agree about google, is nightmare.

If you wanna jump to a FTP Solution, you should evaluate the cost of maintence, because there will be all the data and you should pretty sure for solvent those cost.
If you decide to retire for a while, have backups, etc.

You will need a last two or tree administrator of trust, the others users, can post into a formal request to have an account and you send them a PM with a generated password.

There will have two folders:
The first one is the official, where only those administrators will publish the files, this way probably of issues or risk of bin laden attacks will be prevented.

The other folder will be for the normal contributors, where can upload updates or new decks, etc. once time tested by the adms, will be on the official folder download. and obviously moving those files to the official one.

I know maybe its a lazy work for the adms, but is the best way to control the correctly functions of the CW.

cya

[Xander9009]

The FTP is already up and running. I haven't given anyone access yet, though. It's still in testing to be sure it'll be stable and the internet cable will be undergoing repair soon. However, I'm using IIS 7's FTP functionality on a Windows 2008 server, so it's free (my dad owns and runs the server).

For the passwords, I'm still looking into the ability to set or modify passwords through a program. If I can, then I will make a system so users can change their own. The FTP will only be able to be modified by those with a password. Others will be able to view it (unless it's too popular and it messes with the connection's bandwidth), but not upload files without an account. With that, it's not really necessary to have two separate folders.

The FTP doesn't accept commands to delete files or folders. Only I can do that. Users can add files, but only of certain types.

So, very good advice, if only I wasn't already past that stage. I am glad that everyone is okay with the FTP idea, though. I was worried there'd be a lot more resistance. Once I'm certain it'll be available consistently, I'll start getting in touch with people to make the move.

---------------------

In other news, today, I set up another method of accessing and editing the files. It won't handle the images like the FTP will because it's a website whose sole purpose is editing code in a directory of files. Open source things are amazing. It looks nice and points to the same files as the FTP, so they work together nicely. Those making files through Notepad++ and using its built-in FTP plugin, those using the site, and those using something else altogether all mesh very nicely. I'm rather pleased with how it's going. You can get a look at what it looks like here: http://codiad.com/ Just click Live Demo to see it in action. With that system, users can't make their own passwords, but they can change their own passwords. I've modified the website to remove the option to delete directories. I've left in the option to delete files for now, because unlike FTP, if you're going to use the website to delete thousands of files, it's going to take you a LONG time, and it'll take me two minutes to reupload them.

Before I open it up, though, I want to see about making my laptop mirror the server's directories. My dad had the idea of using my laptop as a backup, and I really like the plan. If for some reason the server goes down, I can start up 3 programs and everything is then running from my laptop temporarily.

One thing I'm particularly excited about is that the fact that there is a server with a URL I can rely on means I can make the tools that update the CW for users work directly. Download a list of changes and download files based on that rather than being forced to run Google Drive if you want to stay automatically updated. So, I think once it's all running I might get back to what really started the whole Community Wad: the tools.

[Xander9009]

Just wanted to let you guys know that this idea is definitely not dead. We've got both the FTP and the Codiad website up and running and all of the cards, functions, and other files are editable through both. You can edit everything either in your browser, or through Notepad++ using its built-in FTP plugin, or download them, modify them, and then upload them using any FTP client you want.

It's secured by restricting the file types that can be uploaded (and I'll be adding in a better backup ability soon, hopefully). I've yet to discuss one of the problems I hadn't foreseen, but hopefully it won't be a big deal. (Repacking requires a logged on user, but it's a server... It runs most of the time with no one logged on... At worst, I can download the files and pack them each night using my laptop like before, but I'd rather make my laptop unnecessary.)

We're currently letting it sit for a few days to make sure it stays up and running. The ISP was having some difficulties, but they should be fixed now (a bad line was running to the house). With any luck, I'll start sending out info in a couple of weeks.

[Zambooo]

What about synchronization? With Google Drive I could just open it and it would download/update everything in the Drive folder, where I could search for names, text or illus with another program (cause having these stuff easily accessible makes coding easier for me). Will I be able to do something like that with FTP?

[Xander9009]

What about synchronization? With Google Drive I could just open it and it would download/update everything in the Drive folder, where I could search for names, text or illus with another program (cause having these stuff easily accessible makes coding easier for me). Will I be able to do something like that with FTP?

Using an FTP client, you can search the folders using an FTP client without even waiting for it download (FileZilla's FTP client at least has this option, I don't know if it's standard). You probably won't be able to search inside of files, though.

However, there are FTP syncing tools. Instead of going through Google Drive, they'd use FTP, but it would be the same idea. I haven't tested any, though. And if all of the free options are terrible, I can make my own specifically for this. It wouldn't be as polished as the others available, but it would work for our purposes.