Firstly, it will refuse to run for anyone other than a local Galaxy administrator. This is because it exposes unrestricted scripting so should only be installed if you can trust your administrative users not to run "cd /; rm -rf *". I'd advise installing ONLY on your own private instance and NEVER on a public Galaxy.
Secondly, it has two modes of operation - script running and tool generation.
When executed without the option to generate a tool archive, it will run a pasted (perl, python, R, bash) script creating an output in the history. This history output is re-doable in the usual Galaxy way including allowing the script to be edited and rerun, so it's possible to (eg) get a script working interactively - galaxy as an IDE anyone ? :)
Once a script runs on some test data, the tool factory will optionally generate a complete tool shed compatible gzip which can be uploaded to any tool shed as a new or updated repository. The generated tool includes the supplied test data as a proper Galaxy functional test. Once a tool is in a toolshed, it is just another Galaxy tool, ready to be installed to any Galaxy like any other tool - but will require restarting of multiple web processes as John mentions.
If the script is safe, the tool is safe - there are no specific security risks for tool factory generated tools other than the script itself.