March 2013 - galaxy-dev - lists.galaxyproject.org

Problem running ncbi_makeblastdb as real user in cluster
by Raj Ayyampalayam 20 Mar '13

20 Mar '13

Hello, I am setting up a local galaxy installation and using our local SGE cluster to run the jobs as real users. One of the first few tools I tested was the ncbi makebkastdb tool. When I try to run it the history for that particular job is immediately shown as follows: 0: (unnamed dataset) Failed to retrieve dataset information. An error occurred with this dataset:/hasattr(): attribute name must be string/ In the stdout of the job I was getting the following error: Fatal error: Matched on Error: Error: NCBI C++ Exception: "/usr/local/src/ncbiblast+/2.2.27/c++/src/objtools/blast/seqdb_writer/build_db.cpp", line 979: Error: ncbi::s_CreateDirectories() - Failed to create directory 'dataset_38_files' I finally figured out that the makedb wrapper was using $outfile.extra_files_path instead of $outfile.files_path. I fixed the wrapper and restarted the instance. The job now runs ok and reports successful completion in the stdout. But, the history for this particular job is still shown as explained above. Can somebody help me figure out what is going wrong in my setup? Here are some of the relevant logs: ==> web1.log <== 128.192.203.31 - - [19/Mar/2013:15:14:35 -0400] "POST /tool_runner/index HTTP/1.1" 200 - "http://galaxy.qbcg.uga.edu/tool_runner?tool_id=toolshed.g2.bx.psu.edu/repos…" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.172 Safari/537.22" ==> web0.log <== 128.192.203.31 - - [19/Mar/2013:15:14:36 -0400] "GET /history HTTP/1.1" 200 - "http://galaxy.qbcg.uga.edu/tool_runner/index" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.172 Safari/537.22" galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,161 Error in history API at listing contents with history d413a19dec13d11e, hda e89067bb68bee7a0: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,316 Error in history API at listing contents with history d413a19dec13d11e, hda ba03619785539f8c: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,316 Error in history API at listing contents with history d413a19dec13d11e, hda cbbbf59e8f08c98c: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,317 Error in history API at listing contents with history d413a19dec13d11e, hda 964b37715ec9bd22: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,317 Error in history API at listing contents with history d413a19dec13d11e, hda 1fad1eaf5f4f1766: hasattr(): attribute name must be string galaxy.webapps.galaxy.api.history_contents ERROR 2013-03-19 15:14:36,318 Error in history API at listing contents with history d413a19dec13d11e, hda 2fdbd5c5858e78fb: hasattr(): attribute name must be string ==> manager.log <== galaxy.jobs.manager DEBUG 2013-03-19 15:14:40,856 (42) Job assigned to handler 'handler0' ==> handler0.log <== galaxy.jobs DEBUG 2013-03-19 15:14:41,790 (42) Working directory for job is: /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 galaxy.jobs.rules.200_rules DEBUG 2013-03-19 15:14:41,791 toolshed.g2.bx.psu.edu/repos/devteam/ncbi_blast_plus/ncbi_makeblastdb/0.0.1 galaxy.jobs.handler DEBUG 2013-03-19 15:14:41,792 dispatching job 42 to drmaa runner galaxy.jobs.handler INFO 2013-03-19 15:14:41,882 (42) Job dispatched galaxy.tools DEBUG 2013-03-19 15:14:42,068 Building dependency shell command for dependency 'makeblastdb' galaxy.tools WARNING 2013-03-19 15:14:42,068 Failed to resolve dependency on 'makeblastdb', ignoring galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:42,453 (42) submitting file /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/pbs/galaxy_42.sh galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:42,453 (42) command is: makeblastdb -version &> /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/tmp/GALAXY_VERSION_STRING_42; makeblastdb -out /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/dataset_41_files/blastdb -hash_index -in " /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/files/000/dataset_30.dat " -title "wert" -dbtype nucl; cd /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist; /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/set_metadata.sh ./database/files /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 . /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/universe_wsgi.ini /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/tmp/tmpxi116L /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/galaxy.json /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_in_HistoryDatasetAssociation_40_4YjnIJ,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_kwds_HistoryDatasetAssociation_40_fWHejS,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_out_HistoryDatasetAssociation_40_BwBD1y,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_results_HistoryDatasetAssociation_40_0IjMAn,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/galaxy_dataset_41.dat,/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/metadata_override_HistoryDatasetAssociation_40_pXhQCd galaxy.jobs DEBUG 2013-03-19 15:14:42,456 (42) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 raj76 1083 galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:42,766 (42) submitting with credentials: raj76 [uid: 740] galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:42,774 (42) Job script for external submission is: /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/pbs/42.jt_json galaxy.jobs.runners.drmaa INFO 2013-03-19 15:14:43,198 (42) queued as 2246095 galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:43,420 (42/2246095) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:14:54,444 (42/2246095) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2013-03-19 15:15:04,614 (42/2246095) job left DRM queue with following message: code 18: The job specified by the 'jobid' does not exist. galaxy.jobs DEBUG 2013-03-19 15:15:04,648 (42) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 galaxy 1234 galaxy.jobs DEBUG 2013-03-19 15:15:06,988 (42) Changing ownership of working directory with: /usr/bin/sudo -E scripts/external_chown_script.py /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 galaxy 1234 galaxy.jobs DEBUG 2013-03-19 15:15:07,833 finish(): Moved /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/galaxy_dataset_41.dat to /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/files/000/dataset_41.dat galaxy.tools DEBUG 2013-03-19 15:15:09,546 Error opening galaxy.json file: [Errno 2] No such file or directory: '/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42/galaxy.json' galaxy.jobs DEBUG 2013-03-19 15:15:09,626 job 42 ended galaxy.datatypes.metadata DEBUG 2013-03-19 15:15:09,626 Cleaning up external metadata files galaxy.objectstore CRITICAL 2013-03-19 15:15:09,665 /panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42 delete error [Errno 39] Directory not empty: '/panfs/pstor.storage/home/qbcglab/galaxy_run/galaxy-dist/database/job_working_directory/000/42' ==> web1.log <== sqlalchemy.pool.QueuePool.0x...4350 WARNING 2013-03-19 15:16:14,801 Error closing cursor: current transaction is aborted, commands ignored until end of transaction block Thanks, -Raj

3 8

What are wrappers, how can I use this?
by Ramon Tiburski 20 Mar '13

20 Mar '13

Hi, I installed galaxy locally and want to develop some tools, but I don't know where start. I would know what are wrappers and what I could do with this, and if is possible develop a tool in java. Thanks Ramão

2 2

Some problems setting up a new cloudman instance on AWS
by Adam Cohn 20 Mar '13

20 Mar '13

Hi Galaxy Admin list, This is my first post to this list. Galaxy is a great tool and I am really enjoying working with it. Background, I am setting up a private galaxy instance for my company using AWS. The installation is going very smoothly with a few problems listed below. I apologize if these questions have been asked before. I have been through the documentation and searched the list pretty thoroughly and can't find the answers. Thanks in advance for your help. Question 1: Can't install tools in the tool shed I have added this line to the universe_wsgi.ini tool_config_file = tool_conf.xml,shed_tool_conf.xml Which references a file that contains this tag: <toolbox tool_path="../shed_tools"> </toolbox> I have created this directory at the correct level and set permission to 755. But I am still getting a non specific error loading tools. I saw that this was (is?) a known issue, but that it had been resolved in the latest build. What am I missing? The log does not give me any more specific information about the error. Any suggestions? Question 2: Weird log error. Every minute the log records this error: galaxy.web.framework DEBUG 2013-03-07 13:07:42,207 Error: this request returned None from get_history(): http://127.0.0.1:8080/root 127.0.0.1 - - [07/Mar/2013:13:07:42 +0000] "GET /root HTTP/1.1" 200 - "-" "Python-urllib/2.6" Is this a known issue? How can I find more information about what is causing it? Question 3: Correct Settings: My intent is to allow employees to access galaxy remotely without allowing public access. In order to keep the data private, I have set: require_login = True allow_user_creation = False Other than the id_secret setting (which I am currently researching) are there any big security holes I am missing with the default installation? Thanks again for the help. Adam

2 5

show urls in csv (in eye mode)
by Hagai Cohen 20 Mar '13

20 Mar '13

Hi, I have a tool which outputs a csv file with links. When I press the eye icon - I would like these links to be shown as links in the browser. It is possible? Thanks, Hagai

2 5

Issue with viz of mapped data in Trackster
by Joachim Jacob | VIB | 20 Mar '13

20 Mar '13

Hi all, Since the last updata of Galaxy, I was not able anymore to view newly added tracks to existing or newly created visualisations. In the webhandler, I have traced down this error: 193.191.128.2 - - [14/Mar/2013:11:02:13 +0200] "GET /api/datasets/407a09c0b5976b4e?hda_ldda=hda&data_type=converted_datasets_state&chrom=chr4 HTTP/1.1" 500 - "http://galaxy.bits.vib.be/visualization/trackster?id=1ec2297821c7f2cb" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.22+ (KHTML, like Gecko) Chromium/17.0.963.56 Chrome/17.0.963.56 Safari/535.22+" galaxy.webapps.galaxy.api.datasets ERROR 2013-03-14 11:03:02,409 Error in dataset API at listing contents: : Traceback (most recent call last): File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 41, in show is_true( kwd.get( 'retry', False ) ) ) File "/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/api/datasets.py", line 96, in _converted_datasets_state if not data_provider.has_data( chrom ): File "/home/galaxy/galaxy-dist/lib/galaxy/visualization/data_providers/genome.py", line 814, in has_data st = summary_tree_from_file( self.converted_dataset.file_name ) File "/home/galaxy/galaxy-dist/lib/galaxy/visualization/tracks/summary.py", line 114, in summary_tree_from_file st = cPickle.load( st_file ) EOFError Thanks for any assistance, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

2 4

Different display ratio (Tools, Center & History)
by Hagai Cohen 20 Mar '13

20 Mar '13

Hi, How is it possible to change the deafult display ratio in galaxy? (The ratio between the tools, center & history) Thanks, Hagai

1 0

Exporting histories fails: no space left on device
by Joachim Jacob | VIB | 19 Mar '13

19 Mar '13

Hi all, Exporting histories fails on our server: " Reason: *Error reading from remote server". *When looking at the logs and the system: tail /var/log/messages Mar 19 15:52:47 galaxy abrt[25605]: Write error: No space left on device Mar 19 15:52:49 galaxy abrt[25605]: Error writing '/var/spool/abrt/ccpp-2013-03-19-15:52:37-13394.new/coredump' So I watched my system when I repeated the export, and saw that Galaxy fills up the root directory (/), instead of any temporary directory. Somebody has an idea where to adjust this setting, so the export function uses any temporary directory? Thanks, Joachim -- Joachim Jacob Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib

2 1

adding new tools and wrapper validation!
by Hakeem Almabrazi 19 Mar '13

19 Mar '13

Hi, I am sure these questions had been asked before but could not find the right answer or did not look enough :) . I just want to add new aligner tool wrapper into local galaxy. I used the existing wrappers (i.e bwa) as template and then modified it accordingly. Here are what I did: Create an xml file and put it to "~tools/sr_mapping/mytool.xml" * Created a py file and added it to the same folder "~tools/sr_mapping/myTool.py" * Configured the tool_conf.xml file * Restarted my galaxy server However, the new tool did not show up in the GUI. Is there anything else I am missing? Is there a validation step I need to perform for both files? I appreciate your help. Regards

2 1

Deletion of items in flexbar repository
by Johannes Röhr 19 Mar '13

19 Mar '13

Hello! Yesterday I updated the Flexbar tool definition in the Galaxy Tool Shed repository flexbar to work with the new version 2.33 of the program. After upload, a message indicating an internal server error showed up. Subsequently, I retried it with a tarball which did not work out either. I tried it several times - still with errors - and reset all repository metadata. In the end I arrived to upload the two new files via hg push on command line. They are now shown in Galaxy tool shed. Nevertheless, as part of the whole process (probably tarball upload) a subfolder flexbar has been created that contains four files. I would like to delete this subfolder including contained files and keep only the two files in the root folder of the flexbar repo. I tried to delete the subfolder without success. The message "No changes to repository." appears. Any idea how I could resolve the issue? Thanks for any suggestions!

2 1

Problems with Galaxy-Project since 3/14/2013 update
by Zain A Alvi 19 Mar '13

19 Mar '13

Dear Sir or Madam, I hope this reaches you well. I am a graduate student at Seton Hall University and it has been a wonderful experience using Galaxy Project's main server (https://main.g2.bx.psu.edu/) for the past few months. Unfortunately, I am experiencing trouble since the update to the servers on Thursday 3/14/2013. I am facing the following problems: 1) Only the http addresses only work for uploading any file from NCBI. In the past, I could use the NCBI FTP addresses to upload the file without any trouble. 2) Unfortunately, none of the online tools seem to run. All of my tasks state that they are "waiting to run." This message has been there since yesterday. The tools that I used since the update are cufflinks and cuffdiff. These tools were working before the update. I was wondering, if you could kindly look into this. I would really appreciate it. Sincerely, Zain Alvi --- PhD Student in Molecular BioSciences Seton Hall University email: zain.alvi(a)student.shu.edu Ph: (973) 710-7149

2 1