Page 2 of 10

Re: Implemented unrar and cleanup script

Posted: Fri Jan 14, 2011 10:53 pm
by mcmanuf
Hello thanks for the quick reply. Looks beautiful.
Do you have any idea why the script sometimes does not run at all after a torrent has been downloaded?

Re: Implemented unrar and cleanup script

Posted: Sat Jan 29, 2011 4:28 pm
by blacke4dawn
Sorry for the late reply, but no I have no idea why. Personally I do not use any download scripts.

Re: Implemented unrar and cleanup script

Posted: Sun Jan 30, 2011 1:26 pm
by mcmanuf
Hello.
This is the unrar/unzip code i want to use.
Can someone tell me what is wrong with the code?

Code: Select all

#! /bin/sh

# posttorrent.sh by Killemov
cd $TR_TORRENT_DIR
if [ -d "$TR_TORRENT_NAME" ]
then
  if ls "$TR_TORRENT_NAME"/*.rar > /dev/null 2>&1
  then
    find "$TR_TORRENT_NAME" -iname "*.rar" | while read file
    do
       unrar e -inul "$file" /mnt/usb2/Downloads/$TR_TORRENT_NAME/ 
	done
     transmission-remote localhost:9091 -n admin:1234 -t$TR_TORRENT_ID --remove &
    echo "Unrarred" $TR_TORRENT_NAME in this dir: $TR_TORRENT_DIR  >> /var/log/posttorrent.log
  elif ls "$TR_TORRENT_NAME"/*.zip > /dev/null 2>&1
  then
    find "$TR_TORRENT_NAME" -iname "*.zip" | while read file
    do
      unzip "$file" -d /mnt/usb2/Downloads/$TR_TORRENT_NAME/
    done
    transmission-remote localhost:9091 -n admin:1234 -t$TR_TORRENT_ID --remove &
    echo "Unzipped $TR_TORRENT_NAME" in this dir: $TR_TORRENT_DIR >> /var/log/posttorrent.log
  fi
fi

Re: Implemented unrar and cleanup script

Posted: Sun Jan 30, 2011 2:12 pm
by blacke4dawn
Code looks fine to me so do you get any errors messages?

Have you tried running it manually? That can be done by setting all the TR_TORRENT_* variables yourself in a terminal window and then just executing the script there.

Re: Implemented unrar and cleanup script

Posted: Sun Jan 30, 2011 9:56 pm
by mcmanuf
Hello now somehow the script started working again!!
The problem seems that the script cannot extract files that contains spaces.
Fx it cannot extract: Test file 1.rar
But it can extract: Test_file_1.rar

How can this problem be adressed?

Furthermore the other problem is that the script cannot deal with rar files wth crc errors!
When tring to extract som rar files it unrar says CRC failed and it cannot extract the archive.

Is it somehow possible to skib this CRC check?
On my PC the winrar version 3.93 extracts all files without problems. The unrar version on the nas is 3.92 beta 1 freeware.

Thanks in advance!

Re: Implemented unrar and cleanup script

Posted: Mon Jan 31, 2011 12:09 am
by blacke4dawn
Considering that we are using quotes around the variable containing the file name it shouldn't be a problem. However it could be the "while read file" that takes a hit, one way to test it would be to replace:

Code: Select all

    find "$TR_TORRENT_NAME" -iname "*.rar" | while read file
    do
       unrar e -inul "$file" /mnt/usb2/Downloads/$TR_TORRENT_NAME/ 
   done
with:

Code: Select all

    find "$TR_TORRENT_NAME" -iname "*.rar" -exec unrar e -inul "{}" /mnt/usb2/Downloads/$TR_TORRENT_NAME/ \;
As for the CRC check, I don't think you can remove it but you can tell it to keep broken files by adding -kb right after -inul (with a space between). Also to see what errors, if any, unrar produces remove -inul.

Re: Implemented unrar and cleanup script

Posted: Mon Jan 31, 2011 6:55 pm
by mcmanuf
Thank you very much for your help.

For now i have discovered a new error that I am experiencing.

I have tried downloading a file called:
"test_a.zip"
"test_b.zip"

What happens is that ONLY "test_a.zip" is extracted.
"test_b.zip" is left untouched by the script!
The error must be within these lines:

Code: Select all

   find "$TR_TORRENT_NAME" -iname "*.zip" | while read file
    do
      /opt/bin/unzip -q "$file" -d /mnt/usb2/Downloads/$TR_TORRENT_NAME

Re: Implemented unrar and cleanup script

Posted: Mon Jan 31, 2011 10:07 pm
by blacke4dawn
Ok I was experimenting a bit with optimization and a "new" version might solve that, seems I was right that the piping to while had something to do with your problems.

Code: Select all

#!/bin/bash
#
# Simple extraction script for torrents with compressed files.

# Directory to store the un-compressed files in.
DEST_DIR="${TR_TORRENT_DIR}/${TR_TORRENT_NAME}"
# Source directory, should not be changed.
SRC_DIR="${TR_TORRENT_DIR}/${TR_TORRENT_NAME}"
# Log file, file where we tell what files have been processed.
LOG_FILE="/var/log/unpacks.log"
# Get current time
RUN_AT=$(date +"%F %T")

if [ -d "${SRC_DIR}" ]; then
  find "${SRC_DIR}" -iname "*.rar" -execdir unrar e -inul -o+ -kb '{}' "${DEST_DIR}" \; -printf "${RUN_AT}: Unpacked %f into %H\n" , -iname "*.zip" -execdir unzip -qq -o '{}' -d "${DEST_DIR}" \; -printf "${RUN_AT}: Unpacked %f into %H\n" >> "${LOG_FILE}"
fi
A few things to consider:
It places the extracted file in the same dir as the compressed ones by default, replace ${TR_TORRENT_DIR} inside DEST_DIR with your preferred destination.
It produces no errors or other output than the simple statements on what compressed files were unpacked. To get more output remove -inul or -qq respectively.
It overwrites without asking. To disable remove -o+ or -o respectively, might not be the best idea though.
Unrar will keep broken file, a.k.a those that fail CRC checks. To disable this remove -kb.
Unlike previous version, this one won't remove the torrent from transmission, didn't find a good way to check if an unpack had actually occurred.


And to ease your mind, I tested it on a directory which had 1 rar-file and 2 zip-files, all 3 were extracted on same run.

Re: Implemented unrar and cleanup script

Posted: Tue Feb 01, 2011 6:20 pm
by mcmanuf
You are a genius sir.
I have testet the script and it now does extract both zip files.

I still have a problem with a single rar file with spaces I just can't get to work with transmission. Will you test it out for me?

It does although work if i run the script on it directly!! hmm...

Re: Implemented unrar and cleanup script

Posted: Wed Feb 02, 2011 5:44 pm
by blacke4dawn
Well, I try my best.

Hmm, sounds very strange that it does work when run manually but not when run by transmission. However there shouldn't be any problems since all those parts are quoted.

Could you say which torrent you're trying to download since finding one would probably take too long, and all my testing have been by running it manually. Might be best via PM perhaps depending on the "legality" of it.

Re: Implemented unrar and cleanup script

Posted: Thu Feb 03, 2011 12:14 am
by blacke4dawn
*sigh and slaps self*
Ok, now I know exactly why it won't unpack that particular torrent and it's because it only looks in subdirectories, that is if the torrent is one single file then this script will ignore it.

A slightly revised script to "fix" that:

Code: Select all

#!/bin/bash
#
# Simple extraction script for torrents with compressed files.

# Directory to store the un-compressed files in.
DEST_DIR="${TR_TORRENT_DIR}/${TR_TORRENT_NAME}/"
# Source directory, should not be changed.
SRC_DIR="${TR_TORRENT_DIR}/${TR_TORRENT_NAME}"
# Log file, file where we tell what files have been processed.
LOG_FILE="/var/log/unpacks.log"
# Get current time
RUN_AT=$(date +"%F %T")

find "${SRC_DIR}" -iname "*.rar" -execdir unrar e -inul -o+ -kb '{}' "${DEST_DIR}" \; -printf "${RUN_AT}: Unpacked %f into %H\n" , -iname "*.zip" -execdir unzip -qq -o '{}' -d "${DEST_DIR}" \; -printf "${RUN_AT}: Unpacked %f into %H\n" >> "${LOG_FILE}"
Unfortunately I ran into another problem when running that script manually and that was that unrar said there was no files to extract (complete manual extraction worked so no fault with archieve). Trying to track that one down now so we will see.

EDIT: Ok, found out that unrar needs a trailing slash to create the destination folder for it to unpack it. Script above updated to reflect it.

Re: Unrar and cleanup script - UPDATE 2011.02.20

Posted: Sun Feb 20, 2011 9:17 am
by killemov
I have updated the start post with a new script.

Changes:
  • It looks for a file named "exit" and if it exists the script will exit.
  • It looks for a file named "keep" and if it exists the torrent will not be removed from transmission.
  • Added a sleep option that will keep the torrent seeding for an hour.
  • Added support for files with whitespace in their name.
Note: There is no support for torrents with a single rar file. (No folder.) These are very rare.

Re: Unrar and cleanup script - UPDATE 2011.02.20

Posted: Fri Apr 22, 2011 9:52 pm
by ambotlance
Great script.
It works wonders!

If i have torrents that I would like not to be deleted from transmission after extraction, could you alter the script to check if the torrent has already been extracted?

Thanks a bunch!

Re: Unrar and cleanup script - UPDATE 2011.02.20

Posted: Sat Apr 23, 2011 12:17 pm
by killemov
ambotlance wrote:If i have torrents that I would like not to be deleted from transmission after extraction, could you alter the script to check if the torrent has already been extracted?
killemov wrote:
  • It looks for a file named "exit" and if it exists the script will exit.
  • It looks for a file named "keep" and if it exists the torrent will not be removed from transmission.
So, just create a file named "keep" (touch keep) in the folder that was created when the torrent was added.
The "exit"(touch exit) option is for when there's a rarred subtitle file present that you don't want processed and you want to keep your downloaded data.

Re: Unrar and cleanup script - UPDATE 2011.05.09

Posted: Mon May 09, 2011 12:48 pm
by killemov
I have updated the start post with a new script.

The only change I made was to check if unrar exits with an error code and handle it by exiting.

There are some weird rare occasions that tm thinks it's all done but the content actually is not complete or has minor corruption.