GTRW2 mod deduplicator

0. Download
1. What is it?
2. Pros & Cons - (why) should I use it
3. Requirements
4. How does it work
5. Introduction - Fast start
6. In Detail
7. FAQ
A. Bugs
B. To Dos
C. Versions - Change Log
D. Legal

0. Download

Please read topics 1 to 5 once before using
Download v1.10
Mediafire Download v1.10



1. What is it?

GTRW2 is a file de duplicator combined with a mod manager. Its eats folders with files, stores unique files in an defined area and keeps track what was in the folder originally included and
sorts it in a category. On demand you can search for the folder (by category or name) and extract it to get the original back.

A quick example, you have 2 folders with the same mod but in different version. The app will store only unique files and the name of the folders.
You will save the amount of the duplicate files and the hassle to sort it.
If you by mistake add again the same mod, the app will not add any new files, just adds an entry.

This is an WiP APP, it works, and will have an upward compatibility.

Be warned, this tool is powerful, can lead to data loose. After every action, please check the status window for errors.


2. Pros & Cons - (why) should I use it

This tool can help you, if you are working with many duplicating files (and have no time to sort) or you running out of space
This tool can help you, if you lost with your files and are unable to find something
This tool can help you, if you want to have make snapshots of your game (without been penalised for hard disc usage)

You should not use this tool, if your packages/mods consist of big single files (like rF2 mods or EXE) or seldom duplicating

You can use it for everything, what is not daily needed and needs somehow sorting


Pros

sort your data/mods
place independent, you can work with relative paths and move so your library to another drive/computer
individual categories
store where & what you want
no install

Cons

it anonymizes the files (you can't recover the data if the database is lost)
it stores the data unpacked (use folder compression to overcome this mostly)
uses a database


3. Requirements

Right now, it needs an running mySQL server to store the data
It was tested on an Win64 environment, but should work on any Windows
It needs an Graphic Adapter supporting OpenGL


4. How does it work

After setting up the App, you show it to a place where you stored you decompressed Mods, each in one folder.
It reads the files in the folder and compares if the file is already in the STORAGE.
If no, it will copied inside and indexed.
If yes, the MOD will use the former file stored when extracted.
After worked through the files, the gathered info is stored in the database under the name of the folder.
On extracting, the DATABASE looks which files are included in the Mod.
There are copied to the extracting path inside a folder called like the mod name.



5. Introduction - Fast start

5.1. Preparation
Decompress the package where you want (if you want to store the data relative, check if the drive has enough free space).
Edit your categories in the types.txt file. Each line is the name of a category (no empty line at the end).
You can change the categories-names afterwards and add more. Categories could be different games or mod types or locations or anything.
Start your mySQL.
Start the App (you can create a link to the desktop, but the home directory needs to be setup).

Generate an Database Entry

[You must be registered and logged in to see this image.]
press DATABASE
[You must be registered and logged in to see this image.]
press CREATE NEW mySQL DB

Edit settings for your mySQL DB.
If you use XAMPP you just need to check the store path where your files should be copied, but you can use store/mods for your first database

[You must be registered and logged in to see this image.]
press CREATE DB

If no errors (see STATUS window) we can go on activate the created DB


[You must be registered and logged in to see this image.]
press CHANGE DB SOURCE

[You must be registered and logged in to see this image.]
select database
[You must be registered and logged in to see this image.]


5.2. Add some Packages

[You must be registered and logged in to see this image.]
Prepare a folder with packages you want to store, remember the folder names will be taken as package names, so name them sensible (like "GTR2 v1.1 STEAM 2013.09.11 Snapshot")

[You must be registered and logged in to see this image.]
press ADD

[You must be registered and logged in to see this image.]
Edit the directory path to the path where your packages are.
The App will scan the path immediately after typing and show all directories.

[You must be registered and logged in to see this image.]
choose the category the packages should be stored as

[You must be registered and logged in to see this image.]
press ADD MODS
After some time the result is showed
[You must be registered and logged in to see this image.]

5.3. Extraction

[You must be registered and logged in to see this image.]
press MODS

[You must be registered and logged in to see this image.]
press GRAB

a list of first 30 entries will show up (depending on your filter)

[You must be registered and logged in to see this image.]
select desired package

[You must be registered and logged in to see this image.]
change destination path

[You must be registered and logged in to see this image.]
press EXTRACT

[You must be registered and logged in to see this image.]
done extracting
[You must be registered and logged in to see this image.]




6. In Detail

6.1. Setting Up

6.1.1. mySQL
If you do not use mySQL until now and have no idea how to set it up, there is an package called XAMPP on the internet (http://www.apachefriends.org/en/xampp-windows.html)
It is a full Apache/PHP/Perl/mySQL package preconfigured. Just install it and start mySQL with the XAMPP control panel and that’s it.

6.1.2. The category file
Until now, there is one category file for all databases. Each line is the name of a category.
If you need different categories for different databases, then you need another instantation of this app.
If you use non ASCII chars, this file needs to be saved as UTF8


6.2. Adding MODS
6.3. Mod handling
6.4. Extracting
6.5. Database

6.5.1

6.5.2

6.5.3 STORAGE
When inserting a path for store the files, you can use a relative path (i.e. store/mods), then the files will be stored inside the app folder.
If you want to store the data on a certain position, you can do so by writing the complete path (i.e. D:\MY MODS ).
You can always move the data, but you need to change the database settings to show where the data is stored.

You should not mix two database storages. The result is fatal.

For performance reasons, you should use another drive for the STORAGE as the source or extraction path. Remember, the files will be copied.

6.6. Settings

7. FAQ

7.1. How to change predefined path (in the add path)? Check the Language Text File. Some of the defined textes are there. Change it and save.
7.2. Can i start the App twice? Yes, but adding and/or deleting on the same database can lead to unpredicted results.

A. Bugs

A.1. The GUI does not allow usage of the ALT GR key. If you need a char which can be created only by this key, then CUT&PASTE it. After pressing ALTGR you need to refocus the Box to be able to write something.
A.2. The Window can not be resized
A.3. ************ fixed in v0.403 During processing, there is no refresh
A.4. ************ fixed in v0.410 Date in log shown is month old
A.5. ************ fixed in v0.414 Big file sizes shows as negative

B. To Dos

B.1. SQLite as alternative for mySQL
B.2. mySQL Database Backup
B.3. Mod Duplicate Console (to find / remove double entries)
B.4. Disable Buttons during long operations

C. Versions - Change Log

v1.101 - 2013.10.20
GUI issues
double starts
changed packeg size to MB

v0.420 - 2013.09.24
mysql problem solved (filenames with ')

v0.414 - 2013.09.17
month fix
md5 fix for files with obscure filename

v0.411 - 2013.09.16
month fix
md5 fix for files with obscure filename

v0.407 - 2013.09.13
changed mod info title row
added % values in add mod window

v0.403 - 2013.09.13
refresh during long operations

v0.401 - 2013.09.13
addapted new GUI version

v0.391 - 2013.09.12
added small overwrite check (mod extraction)
added row info in mod file listbox
added extraction comment in mod listbox (after extraction)

v0.386 - 2013.09.11
first public release

D. Legal

By using, copying or distributing my software and accompanying documentation (the program's object code and documentation are collectively referred to as „the software”), you indicate your acceptance of the following license terms:

Usage
I grant you (the licensee) a permission to use the software at no cost, for non-commercial purposes on any computer or device in your possession. You may also make copies and distribute the software according to requirements of section below.

Warranties and Disclaimer
The software and any support from me are provided “AS IS” and without warranty, express or implied. I specifically disclaim any implied warranties of merchantability and fitness for a particular purpose. In no event will i be liable for any damages, including but not limited to any lost profits, lost savings or any incidental or consequential damages, whether resulting from impaired or lost data, software or computer failure or any other cause, or for any other claim by the user or for any third party claim.

Limit of Liability
Subject to any liability which may not be excluded or limited by law the licensor shall not be liable and hereby expressly excludes all liability for loss or damage howsoever and whenever caused to you.

Distribution
As used in this agreement, the term “distribute” (and its variants) includes making the Software available (either intentionally or unintentionally) to third parties for copying or use.You may not impose any further restrictions on the recipients' exercise of the rights granted herein.

You may distribute the software under the following conditions:
You may not modify, combine other commercial applications with, or otherwise prepare derivative works of the software. You may not distribute the software for a fee (by payment of money or otherwise, whether direct or indirect). In particular (but not limited to), you may not charge for the software to recover media costs associated with its distribution, you may not charge for a product or compilation the software is a part of, you may not provide a paid download of the software and you may not require a registration of some kind (for example, on your website) for downloading the software. you may personally install the software as part of a free or paid service. you may distribute the software along with computer hardware, as long as there is a reference to the inclusion of the software (for example, a printed logo or a desktop icon on preinstalled computer systems). You must not distribute the software in a manner, which arouses the impression that the software cannot be obtained for free or involves fees for its usage.

Thank you. Anything that can help us organizing the mods is very welcome.

Your pics on this post and the download link are "out of reach" for me in Brazil, I had to use a proxy and reload this page and the link in order to download the app. Maybe more people will face the same problem.

 

MacGyver666 wrote:Thank you. Anything that can help us organizing the mods is very welcome.

Your pics on this post and the download link are "out of reach" for me in Brazil, I had to use a proxy and reload this page and the link in order to download the app. Maybe more people will face the same problem.

;) 

Thanks for the feedback.
I will swap the images to another hoster and add an alternative download asap.

Would like to see feedback, before working on big improvements and sqlite support (which will not force mysql usage).

I'm going to test it tomorrow. I'm glad having the XAMPP already on my computer

yuca wrote: Thanks for the feedback.
I will swap the images to another hoster and add an alternative download asap.

Would like to see feedback, before working on big improvements and sqlite support (which will not force mysql usage).

Pics are now showing properly, but the download link is still the same or still out of reach.  

Thanks,

now added an mediafire link, this should help.

08.22.2015As promised, i worked over the program for using as a database sqlite.
Here is my first release of the new program, unfortunately only as command line version. A GUI is in progress. right now, i shifted my GTRW2 mod database to the new one and working well.

Its handling 140.000 Mods & other etries with 3.600.000 files and 40.000.000  different entries.

Main difference between gtrw2 and gtrw3:
switch from mySQL to sqlite
files can be stored in (compressed) archives

How to use it

1. uncompress a bunch of mods (or other file packets) into an target folder (let say into n:\temp)

2. issue following command
   gtrw3.exe -a n:\temp 1 22
   this will handle all folders in n:\temp as separate MODS and archive them as type 1 mods with the fastest compression method


If somebody wants to help me, i am happy to send via pm a precofigured version.
The created Archive will be compatible with future gtrw3 version and therefore it will be safe to work with them in the future.
As soon the app will have a stable gui, a release will follow


link to the alpha version
[You must be registered and logged in to see this link.]

Thanks for this tool

as promised, you can find now a gui version of my app on my website (http://aspotter.system-ns.eu/).
there will be also a more in depth tutorial how to use it.
this is pretty stable version, but still not complete or final.
main handicap of the gui is not the functionality but the low amount of feedback in case of failing
but i will work on it
the archive function is rather stable and i completely switched from gtrw2 to this version

new gui test version ready for download,
for usage take a look on my website
docs & updated version:
[You must be registered and logged in to see this link.]