- galaxy-dev - lists.galaxyproject.org

samtools installation problems
by Nathan Kipniss 02 Jan '15

02 Jan '15

Hi All, New to galaxy here. I'm having an issue uploading bam files into a local instance of galaxy. I understand that it requires samtools, and I installed the appropriate dependency from the toolshed. However, I get the following error: Traceback (most recent call last): File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 407, in <module> __main__() File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 396, in __main__ add_file( dataset, registry, json_file, output_path ) File "/home/galaxy/galaxy-dist/tools/data_source/upload.py", line 324, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 147, in dataset_content_needs_grooming version = self._get_samtools_version() File "/home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py", line 129, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/usr/lib/python2.7/subprocess.py", line 710, in __init__ errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory Which leads me to believe something is wrong with the install (can't find samtools). The dependency directory is also set. Is there another step I'm missing, or is it better to not install dependencies via the toolshed? Best, Nathan Kipniss

4 5

Trouble with async data sources
by Eric Rasche 30 Dec '14

30 Dec '14

1 0

inconsistency in data library display
by Ryan G 23 Dec '14

23 Dec '14

I'm uploading a bunch of fastq.gz files to a data library. I'm currently doing this one file at a time to test some API functionality. I noticed that right after the web page refreshes itself, the file name is displayed with a '.gz' at the end. If I navigate to the data library by clicking on 'Manage Data Libraries", etc, the file is displayed without the '.gz'. Not a big deal, but it was throwing me off for a while trying to figure out how to use the underlying API.

1 1

Feature Request: Ability to you keyword to move through items in the main window similar to Gmail
by Ryan G 23 Dec '14

23 Dec '14

When browsing a data library, clicking on entries I want to select is a bit cumbersome. It would be nice to use arrow keys, etc to select items, just like when selecting mail in Gmail.

2 3

rpy vs rpy2
by Ryan G 22 Dec '14

22 Dec '14

I'm trying to install rpy for Galaxy, however it seems to not be available anymore. rpy2 is supposed to be the replacement. Does/will Galaxy migrate to rpy2? Some of the tools rely on rpy but since its not available anymore, the tools can't run. Also, why use a R python wrapper? Why not just create an R script and run 'Rscript' directly? Ryan

2 2

Create a package with binaries
by KRESS Arnaud (ESP) 22 Dec '14

22 Dec '14

Hi gentlemen, I am currently struggling to create a custom package (to share via a toolshed) that would include a tool definition file and the associated binary. Once installed, I can launch a job but I get the following error message: mytool.sh: line 14: mytool: unknown command It seems that the PATH was not correctly set. What am I doing wrong ? My directory tree in my package: . ├── mytool.sh ├── mytool.xml ├── bin │ └── mytool └── tool_dependencies.xml Here is my tool_dependencies.xml file content: <?xml version="1.0"?> <tool_dependency> <package name="mytool" version="1.0"> <install version="1.0"> <actions> <action type="set_environment"> <environment_variable name="PATH" action="prepend_to">$INSTALL_DIR/bin</environment_variable> </action> </actions> </install> <readme> </readme> </package> </tool_dependency> Thank you, AK

3 4

Problem with LWR (input size limitation)
by Misharl Monsoor 22 Dec '14

22 Dec '14

Hi everybody, In our lab, we are trying to connect our Galaxy instance to a Windows 7 64 bits server, in order to executes programs that need to be run within Windows. However we have a problem concerning the size of the input that is uploaded to the Windows server, it doesn't accept inputs with a size > 1.3 Gb ? We have the following problem with mmap function: data = mmap.mmap(input.fileno(), 0, access=mmap.ACCESS_READ) error: [Errno 12] Cannot allocate memory I have another question concerning the inputs, is there a way to use several inputs that be uploaded to a Windows Server? Thank you very much in advance, Bests regards, Cheers, Misharl Monsoor

2 8

Metadata error in uploading files to data libraries
by Jelle Scholtalbers 19 Dec '14

19 Dec '14

Hi all, after an update to the following changeset(14859:7ba05957588a, stable, 05.12.14), our bam files that are uploaded(linked) to a data library, are no longer indexed. The metadata_xxx.dat is created, but it stays empty. The following error message appears in the log, although the state of the dataset is 'ok': galaxy.jobs WARNING 2014-12-05 12:47:02,218 Error accessing /g/K/K27.bam, will retry: [Errno 1] Operation not permitted: '/g/K/K27.bam' galaxy.jobs WARNING 2014-12-08 13:38:57,045 Error accessing /g/K/K2.bam, will retry: [Errno 1] Operation not permitted: '/g/K/K2.bam' All file permissions are correct (i.e. galaxy owns them). Furthermore, executing samtools index, just works on those files: samtools index /g/K/K2.bam /g/galaxy/galaxy_data/files/_metadata_files/006/metadata_6598.dat When uploading the file - "copy files into galaxy" - the samtools index just works. ============== Now, on a clean local install(14874:885f940bff64, stable, 05.12.14) and samtools installed globally and with the bam file sorted, I get the following situation: When I try to upload this bam to a data library by linking the following error is shown on the dataset (note: here the dataset is set in error state, which does not happen on our server) Uploaded by: scholtal(a)embl.de Date uploaded: Mon Dec 8 17:42:39 2014 (UTC) File size: 2.6 GB UUID: d23cf11a-0372-41cb-939a-7c8761d78b73 Data type: auto Build: ? Miscellaneous information: Traceback (most recent call last): File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py", line 407, in __main__() File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py", line 396, in _ Job Standard Error Traceback (most recent call last): File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py", line 407, in __main__() File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py", line 396, in __main__ add_file( dataset, registry, json_file, output_path ) File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py", line 294, in add_file if datatype.dataset_content_needs_grooming( dataset.path ): File "/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py", line 147, in dataset_content_needs_grooming version = self._get_samtools_version() File "/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py", line 129, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1308, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory error Database/Build: ? Number of data lines: None Disk file: /Users/scholtalbers/workspace/idr_data/WT1.sort.bam =============================== When uploading the bam without linking, I see the following processes: Upload->set meta->samtools index->'error state' Miscellaneous information: uploaded bam file Traceback (most recent call last): File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py", line 407, in __main__() File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.p Job Standard Error Traceback (most recent call last): File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py", line 407, in __main__() File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py", line 396, in __main__ add_file( dataset, registry, json_file, output_path ) File "/Users/scholtalbers/workspace/galaxy-dist-new/tools/data_source/upload.py", line 324, in add_file if link_data_only == 'copy_files' and datatype.dataset_content_needs_grooming( output_path ): File "/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py", line 147, in dataset_content_needs_grooming version = self._get_samtools_version() File "/Users/scholtalbers/workspace/galaxy-dist-new/lib/galaxy/datatypes/binary.py", line 129, in _get_samtools_version output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, stdout=subprocess.PIPE ).communicate()[1] File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__ errread, errwrite) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1308, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory error Database/Build: ? Number of data lines: None Disk file: /Users/scholtalbers/workspace/galaxy-dist-new/database/files/000/dataset_6.dat ================================= Although the error messages are different, it might be related? Cheers, Jelle

2 4

Re: [galaxy-dev] Allow wildcards in upload files from filesystem paths
by Ryan G 19 Dec '14

19 Dec '14

Thanks. It would be a nice feature. I don't have write permissions on the filesystem where the fastq files are stored. On Thu, Dec 18, 2014 at 4:08 PM, Martin Čech <emulatorer(a)gmail.com> wrote: > > Wildcards are not supported in library upload. Sorry. > > You can either specify one by one or you can just move the .log files > outside of a folder for a bit. Once the fastqs are uploaded you can move > them back. > > Martin > > On Thu Dec 18 2014 at 3:46:30 PM Ryan G <ngsbioinformatics(a)gmail.com> > wrote: > >> I have a directory with a bunch of fastq files mixed with .log files. I >> only want to upload the .fastq files. Do I have to specify each fastq file >> separately or can I use '*.fastq'? I tried the later, but got the error >> 'Invald paths'. >> ___________________________________________________________ >> Please keep all replies on the list by using "reply all" >> in your mail client. To manage your subscriptions to this >> and other Galaxy lists, please use the interface at: >> https://lists.galaxyproject.org/ >> >> To search Galaxy mailing lists use the unified search at: >> http://galaxyproject.org/search/mailinglists/ > >

1 0

Job wrapper object
by Alexandre Defelicibus 18 Dec '14

18 Dec '14

Hi all, I'm developing some tools on my Galaxy instance and I need to get some information about the job that a tool creates. The question is, how can I have a job_wrapper object in my tool wrapper? I tried to use the reserved variable $__app__, but I got errors. I couldn't import and use the galaxy lib in order to get the job information. Could someone show me the correct way to do that, with some examples? I appreciate all the help. Best regards, -- Alexandre Defelicibus Mestrando em Bioengenharia Programa de Pós-Graduação em Bioengenharia Universidade de São Paulo - USP

3 4