TYPO free

home

fighting for TYPO free code

TYPO3 housekeeping part 2

26 Jun 2008

So here we are again. I hope you have gotten the hang of cleaning your TYPO3 installation and are reaping the benefits. If you are like me, you dont like to repeat yourself. All this typing on the command line, can't we automate this? The vacume cleaner in the image here cleans your rooms on its own. We would like TYPO3 to do that too. The problem is that we are not sure if we want to really blindly delete and cleanup all the records found by the cleaner scripts. One solution is to run all of them with the --AUTOFIX --dryrun --YES options and capture the output. Then analyse the output and remove the --dryrun option when you are positive that all is well.

So it's time to cut and paste the commands one last time and create a nice TYPO3Cleaner.sh script:

#!/bin/sh
#
# TYPO3 cleaner script
#

php=/Applications/MAMP/bin/php5/bin/php
webroot=/Users/michiel/htdocs/alien
command="$php $webroot/typo3/cli_dispatch.phpsh lowlevel_cleaner"
options=''
autofix='--autofix'
yes='--yes'
dry='--dryrun'

$command orphan_records -r -v 2 -s $options $autofix $dry $yes
$command versions -r -v 2 -s $options $autofix $dry $yes
#$command tx_templavoila_unusedce -r --refindex update -v 2 -s $options $autofix $dry $yes
$command double_files -r --refindex update -v 2 -s $options $autofix $dry $yes
$command deleted -r -v 1 -s $options $autofix $dry $yes
$command missing_relations -r --refindex update -v 2 -s $options $autofix $dry $yes
$command cleanflexform -r -v 2 -s $options $autofix $dry $yes
$command rte_images -r --refindex update -v 2 -s $options $autofix $dry $yes
$command missing_files -r --refindex update -v 2 -s $options $autofix $dry $yes
$command lost_files -r --refindex update -v 2 -s $options $autofix $dry $yes

Don't forget to chmod+x it after creation.

This is a small step in the right direction. But we can do better. We realy don't want to fiddele with the DRY and YES variable every time we run the script.

Here is a more advanced version:

#!/bin/sh
# TYPO3cleaner.sh

php=/Applications/MAMP/bin/php5/bin/php
webroot=/Users/michiel/htdocs/
script="/typo3/cli_dispatch.phpsh lowlevel_cleaner"


# Cleaning commands
clean () {
$command orphan_records -r -v 2 $options
$command versions -r -v 2 $options
$command tx_templavoila_unusedce -r --refindex update -v 2 $options
$command double_files -r --refindex update -v 2 $options
$command deleted -r -v 1 $options
$command missing_relations -r --refindex update -v 2 $options
$command cleanflexform -r -v 2 $options
$command rte_images -r --refindex update -v 2 $options
$command missing_files -r --refindex update -v 2 $options
$command lost_files -r --refindex update -v 2 $options
}


# Usage message
usage () {
echo "Usage:\n`basename $0` [--dryrun] [--autofix] [--yes] [--quiet] [domain-name]"
}
if [ -z "$1" ]
then
usage
exit
fi


# Initialize variables
until [ -z "$1" ]
do
case "$1" in
"--quiet" )
quiet="-ss"
;;

"--dryrun" )
dry='--dryrun'
;;

"--autofix" )
autofix='--AUTOFIX'
;;

"--yes" )
yes='--YES'
;;

[a-z]* )
if [ -e $webroot$1 ]
then
domain=$1
command="$php $webroot$domain$script"
options="$autofix $dry $yes $quiet"
clean
else
echo "Directory does not exist: $webroot$1"
exit
fi

;;
esac
shift
done

Please find a version with pretty colors down in the scripts section.

And here is a version that is cleaner and shorter. It uses getopts to parse the options. Thank you Frans!

#!/bin/sh
# TYPO3cleaner.sh

php=/Applications/MAMP/bin/php5/bin/php
webroot=/Users/michiel/htdocs/
script="/typo3/cli_dispatch.phpsh lowlevel_cleaner"


# Cleaning commands
clean () {
$command orphan_records -r -v 2 $options
$command versions -r -v 2 $options
$command tx_templavoila_unusedce -r --refindex update -v 2 $options
$command double_files -r --refindex update -v 2 $options
$command deleted -r -v 1 $options
$command missing_relations -r --refindex update -v 2 $options
$command cleanflexform -r -v 2 $options
$command rte_images -r --refindex update -v 2 $options
$command missing_files -r --refindex update -v 2 $options
$command lost_files -r --refindex update -v 2 $options
}


# Usage message
usage () {
echo "Usage:\n`basename $0` [-d(ryrun)] [-a(utofix)] [-y(es)] [-q(uiet)] domain-name"
}


# get options
while getopts ':adqy' FLAG
do
case "$FLAG" in
'a') options="$options --AUTOFIX" ;;
'd') options="$options --dryrun" ;;
'q') options="$options --quiet" ;;
'y') options="$options --YES" ;;
'?') usage; exit 1;;
esac
done
shift $(($OPTIND - 1))

if [ -z "$1" ]
then
usage
exit
fi


# Initialize variables
if [ -e $webroot$1 ]
then
domain=$1
command="$php $webroot$domain$script"
clean
else
echo "Directory does not exist: $webroot$1"
exit
fi

So you can see a lot of automation can be done easily. Implementing mail notification and reporting are left as tasks for the reader ;-).

Happy cleaning!

Frans 29 Jun 2008, 11:55
Hi Michiel,

Nice script, apart from the homework part.

Isnt there a flaw? If u dont specify a domain, but some options nothing will happen?

Why dont u use the shell builtin 'getopts'? It nicely checks for none allowed options.

Tnx, Frans
AndrĂ¡s 15 Aug 2008, 12:10
Hi,

this is very nice indeed. I think it's worth mentioning that there is a "KB Cleanup Files" extension in the repository, that will delete unreferenced uploaded objects from the "uploads" directory. This is also a big space saver.

What I was also looking for is a database structure cleanup utility. When I develop my own plugins, I end up modifying the table structures a couple of times. Having removed some fields from table definitions do not remove the fields from the database. Any suggestion how to clean up those unused fields?
georgk 28 Mar 2009, 12:54
Andreas, you might wanna use the builtin db-compare tool, which compares the tca to the db-tables and offers rename and cleanup; it does so for the core and all extensions.
check it out in the db-section of the install-tool.
G.Gothe 16 Apr 2012, 17:29
When it comes to updates on the DB you should take a lookt at t3deploy, see https://forge.typo3.org/projects/show/extension-t3deploy
Commenting is closed for this item