May 2010 - galaxy-dev - lists.galaxyproject.org

Re: [galaxy-dev] [galaxy-user] import local files
by Nate Coraor 04 May '10

04 May '10

Ryan Golhar wrote: > Hi - I recently discovered Galaxy and am just getting started with it... > I installed a local instance to test with NGS data. Hi Ryan, I moved this over to the galaxy-dev list since it involves a local install. > We have a lot of NGS datasets (both Solid and illumina) on a SAN > available via NFS. Each user that has data has an account on the system > in a directory tree such as: > > /ngsdata/user1 > /ngsdata/user2 > /ngsdata/project1 > /ngsdata/project2 > > where user1 and user2 are distinct users with their own dataset, and > project1, project2 contain datasets shared by multiple users. > > I want to import this data in Galaxy and I came across this thread in > the mailing list (quoted below). > > I set up my Galaxy instance to see this NFS share...btw Galaxy is > running as its own user on a virtual machine. The Galaxy user only has > read access to this read and nothing else. If the usernames match your user's email addresses (this could be done with symlinks), users can import their own data, btw. See the user_library_import_dir option. > When I went to import the files by specifying a local system path on the > admin user interface, I got errors importing the data. I checked the > paster.log file and I saw errors related to galaxy trying to change the > permissions of the files to 0644. > > Does this mean all the files need to be owned by Galaxy? No, this would be a bug. There were a number of bugs like this that I fixed a month or so ago. Could you check your Galaxy version? Thanks, --nate > > > > ------------------- > Old thread: > > Hello Andreas, > > The best approach to handle this is to have a Galaxy administrator > upload the files into a library, creating library datasets. Set the > following config setting to point to the NFS accessible directory you > use to contain the files: > > # Directories of files contained in the following directory can be > uploaded to a library from the Admin view > library_import_dir = /var/opt/galaxy/import > > You can set ACCESS permissions on the library to restrict access to > specific users, or leave it public to allow anyone to access the library > datasets. > > Users can import the library datasets into their own histories for > analysis - doing this does not create another disk file, but simply a > "pointer" to the library dataset's disk file. > > Greg Von Kuster > Galaxy Development Team > > > Andreas Kuntzagk wrote: > > Hi, > > > > to reduce traffic when importing big dataset into our local galaxy we > > would like to copy directly from the fileserver (which is accessible > > from the galaxy server via NFS) into galaxy without moving through the > > users desktop computer. > > Is there already such a tool by chance? At a glance I could only find > > import.py which imports only from a predefined set of files. > > So if no such tool exist what would be the best starting point? > > upload.py? > > For security reasons I would restrict the import to a certain directory. > > Anything else to keep in mind? > > > > regards, Andreas > > _______________________________________________ > > galaxy-user mailing list > > galaxy-user at bx.psu.edu > > http://mail.bx.psu.edu/cgi-bin/mailman/listinfo/galaxy-user > > > > > > _______________________________________________ > galaxy-user mailing list > galaxy-user(a)lists.bx.psu.edu > http://lists.bx.psu.edu/listinfo/galaxy-user

2 4

[Fwd: Galaxy tool error report from golharam@umdnj.edu]
by Ryan Golhar 04 May '10

04 May '10

Hi all - I'm trying to plot the data from a fastq file uploaded to my local instance. When I rung the boxplot app on the uploaded file, I get this error. It has something to do with the parameters to plot in gnuplot not being right. I don't know gnuplot well enough (at all actually) so I don't know what is wrong. The error show is in reference to 'tmargin', but actually I think it has something to do with the whiskerbars parameter. How do I know this? When I run: gnuplot> plot '/mnt/ibmfcarray/galaxydata/database/files/000/dataset_46.dat' using 1:7:11:12:9 with candlesticks lt 1 lw 1 title 'Quartiles' I see a plot, but when I add gnuplot> plot '/mnt/ibmfcarray/galaxydata/database/files/000/dataset_46.dat' using 1:7:11:12:9 with candlesticks lt 1 lw 1 title 'Quartiles' whiskerbars, I get the same error. GALAXY TOOL ERROR REPORT ------------------------ This error report was sent from the Galaxy instance hosted on the server "genomes.umdnj.edu" ----------------------------------------------------------------------------- This is in reference to dataset id 50 from history id 2 ----------------------------------------------------------------------------- You should be able to view the history containing the related history item 14: Boxplot on data 10 by logging in as a Galaxy admin user to the Galaxy instance referenced above and pointing your browser to the following link. genomes.umdnj.edu/history/view?id=529fd61ab1c6cc36 ----------------------------------------------------------------------------- The user 'golharam(a)umdnj.edu' provided the following information: ----------------------------------------------------------------------------- job id: 12 tool id: qual_stats_boxplot ----------------------------------------------------------------------------- job command line: gnuplot < '/opt/galaxy_dist/database/job_working_directory/12/tmpnKQZFP' 2>&1 || echo "Error running gnuplot." >&2 ----------------------------------------------------------------------------- job stderr: Error running gnuplot. ----------------------------------------------------------------------------- job stdout: line 0: undefined variable: tmargin gnuplot> plot '/mnt/ibmfcarray/galaxydata/database/files/000/dataset_46.dat' using 1:7:11:12:9 with candlesticks lt 1 lw 1 title 'Quartiles' whiskerbars, '' using 1:8:8:8:8 with candlesticks lt -1 lw 2 title 'Medians', "< python -c \"for xval, yvals in [ ( fields[1 - 1], fields[13 - 1].split( ',' ) ) for fields in [ line.rstrip( '\\n\\r' ).split( '\\t' ) for line in open( '/mnt/ibmfcarray/galaxydata/database/files/000/dataset_46.dat' ) if not line.startswith( '#' ) ] if len( fields ) > max( 1 - 1, 13 - 1 ) ]: print '\\n'.join( [ '%s\\t%s' % ( xval, yval ) for yval in yvals if yval ] )\"" using 1:2 with points pt 29 title 'Outliers' ^ line 0: ';' expected ----------------------------------------------------------------------------- job info: None ----------------------------------------------------------------------------- job traceback: None ----------------------------------------------------------------------------- (This is an automated message). -- CONFIDENTIALITY NOTICE: This email communication may contain private, confidential, or legally privileged information intended for the sole use of the designated and/or duly authorized recipient(s). If you are not the intended recipient or have received this email in error, please notify the sender immediately by email and permanently delete all copies of this email including all attachments without reading them. If you are the intended recipient, secure the contents in a manner that conforms to all applicable state and/or federal requirements related to privacy and confidentiality of such information.

3 3

sharing saved visualization trackster with galaxy user
by Vipin TS 04 May '10

04 May '10

Hi Galaxy team, I had created a visualization tracks using dataset in wiggle variableStep file and saved to galaxy trackster type. then I tried to share with a galaxy user, the galaxy user can access the trackster from "Saved Visualizations" shared visualization section. the main issue is that the tracks are not visible to the shared user. Please let me know something came wrong from my side. thanks for the support, Vipin T S

2 1

Re: [galaxy-dev] Configuring MySQL for Galaxy (and vice versa)
by Jeremy Goecks 04 May '10

04 May '10

Hi Bastien, Please make sure to cc the galaxy-user and galaxy-dev lists as they help your questions reach a wider audience. In fact, I'm sure there are people on these lists that have more DB knowledge than myself. That said, you're spot on in your thinking. Galaxy uses a single database (galaxy, in your case), so you can setup a "Galaxy" mysql user to have rights only on that DB; specifying localhost provides another layer of security as well. Best, J. On May 3, 2010, at 11:08 AM, Chevreux, Bastien wrote: >> From: Jeremy Goecks [mailto:jgoecks@gmail.com] On Behalf Of Jeremy Goecks >> You can specify a MySQL username and password in the MySQL URL, e.g. >> mysql://username:password@localhost:3306/galaxy?unix_socket=/var/run/ >> mysqld/mysqld.sock > > Hello Jeremy, > > thank you very much for that, works like a charm. > > Coming to my next question: how should I setup the user / user rights / databases / tables etc. in MySQL? > > Yes, I could create a user and then do a > > grant all privileges on *.* to someusr@localhost > > and I think that that Galaxy will then create everything by itself. However, on the long term I feel that this might be ... um ... a security risk should MySQL also host other things than just a Galaxy DB. > > Is there a recommended way to setup things? Like perhaps, e.g., having the mysql admin just create needed databases and giving all rights just to these DBs to galaxy like in: > > create database somedb; > grant all privileges on somedb.* to someusr@"localhost" identified by > 'passwrd'; > > and then have Galaxy work with that? But for that I'd need to know which DBs Galaxy uses and I have to set up. > > If not, I'll do the all privileges on *.*, but I don't really like that. > > Regards, > Bastien > > PS: sorry for nagging, but I'd really like to have a basic security level > PPS: if the above looks like straight from some manual, yes it is. I'm no > MySQL expert, but a good Google search brings most of what I need to > know. > > -- > DSM Nutritional Products AG > R&D Human Nutrition & Health > Bioinformatics - Bldg. 203 / 115 > P.O. Box 2676 > CH-4002 Basel / Switzerland > Tel. +41 61 815 8264 > > > DISCLAIMER : > This e-mail is for the intended recipient only > If you have received it by mistake please let us know by reply and then delete it from your system; access, disclosure, copying, distribution or reliance on any of it by anyone else is prohibited. > If you as intended recipient have received this e-mail incorrectly, please notify the sender (via e-mail) immediately.

2 1

Asking for Galaxy main public instance support, thanks!
by WangJun 04 May '10

04 May '10

Hello Dr Greg Von Kuster, We are developing a Rice Genome Browser: Rice-Map. I am writing to ask if you can add our resource to your main public instance so that users can get data from Rice-Map and do analysis in your platform conveniently. Steps for adding Rice-Map: 1. Add tool definition file "cbi_rmap_browser.xml" under tools/data_source directory (attached). 2. Add this tool into file "tool_conf.xml" in the "Get Data" section: <tool file="data_source/cbi_rmap_browser.xml" /> Thank you in advance! Best wishes, Kathleen Center for Bioinformatics Peking University

2 3

Configuring MySQL for Galaxy (and vice versa)
by Chevreux, Bastien 03 May '10

03 May '10

Dear Galaxy devs, I'd like to try a Galaxy installation with a database connection and have it use a freshly installed MySQL. It's a Kubuntu system where I just installed all needed packages, MySQL is up and running fine. In the Galaxy "universe_wsgi.ini" I commented out the sqlite entry and activated the one for a DB connection and it now shows now this: database_connection =mysql:///galaxy?unix_socket=/var/run/mysqld/mysqld.sock database_engine_option_echo = true database_engine_option_echo_pool = true database_engine_option_pool_size = 10 database_engine_option_max_overflow = 20 database_engine_option_pool_recycle = 7200 What I can't find in the online documents (Wiki) or via a Google search is: 1) how to tell Galaxy it should use a given MySQL user, because using "sh run.sh" then stops with these errors: ... galaxy.model.migrate.check DEBUG 2010-05-03 15:15:27,151 MySQL_python egg successfully loaded for mysql dialect 2010-05-03 15:15:27,187 INFO sqlalchemy.pool.QueuePool.0x...4f90 Error on connect(): (OperationalError) (1045, "Access denied for user 'bach'@'localhost' (using password: NO)") None None sqlalchemy.pool.QueuePool.0x...4f90 INFO 2010-05-03 15:15:27,187 Error on connect(): (OperationalError) (1045, "Access denied for user 'bach'@'localhost' (using password: NO)") None None Traceback (most recent call last): File "/bioinf/wwwservices/galaxy/galaxy-dist/lib/galaxy/web/buildapp.py", line 61, in app_factory app = UniverseApplication( global_conf = global_conf, **kwargs ) ... which is somewhat understandable as such user does not exist in the mysql database. I'm currently trying out Galaxy under my own account, in the end however it will run as separate user. 2) perhaps also important: should the mysql admin setup users, tables and other stuff in mysql? If so: which, how ... etc. Regards, Bastien PS: in case it's important: the hg clone is from "galaxy-dist", march 17. -- DSM Nutritional Products AG R&D Human Nutrition & Health Bioinformatics - Bldg. 203 / 115 P.O. Box 2676 CH-4002 Basel / Switzerland Tel. +41 61 815 8264 DISCLAIMER : This e-mail is for the intended recipient only If you have received it by mistake please let us know by reply and then delete it from your system; access, disclosure, copying, distribution or reliance on any of it by anyone else is prohibited. If you as intended recipient have received this e-mail incorrectly, please notify the sender (via e-mail) immediately.

2 1

Galaxy issue
by Modi, Amit 02 May '10

02 May '10

Hi, We are trying to configure Galaxy to work with our tool. So far I have been able to configure it well and link it with the cluster to do process all the jobs as per the documentation . Now what we want is the before the actual processing starts the validations for the input files are done on the Local. And the job processing is done on the cluster. Is there anyway we can configure this. We are using sge on the cluster. Thanks a lot. Regards, Amit Modi ----------------------------------------- Confidentiality Notice: The following mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. The recipient is responsible to maintain the confidentiality of this information and to use the information only for authorized purposes. If you are not the intended recipient (or authorized to receive information for the intended recipient), you are hereby notified that any review, use, disclosure, distribution, copying, printing, or action taken in reliance on the contents of this e-mail is strictly prohibited. If you have received this communication in error, please notify us immediately by reply e-mail and destroy all copies of the original message. Thank you.

5 17

tiny usability issue - cloning vs. importing shared history
by Assaf Gordon 02 May '10

02 May '10

Hi, I've noticed a small usability annoyance with shared histories. Scenario: A user shared a history with me. The new history appears in the "Histories shared with you by others" page. I click on the history name, get a popup menu with three options: View, Clone, Unshare. If I choose "clone", the history is cloned and appears in my "saved history". If I choose "view" (and look at the library) then click on the green import button, I get a red error message: "The owner of this history has disabled imports via this link. You can return to the previous page." I understand that "cloning" is technically different from "import via link", but conceptually, the end result is the same - I should get the shared history into my histories. So it's quite confusing when I get this error message. Since I'm slightly familiar with Galaxy, I realized that the correct thing to do is go back to the list of shared-histories, then "clone" instead of import. An inexperienced user might think that something is wrong with the sharing. Somewhat related: After importing a history via link, the "start using this history" link does not work if the current history a non-empty. To reproduce: 1. go to http://main.g2.bx.psu.edu/ 2. click "options" -> "create new". 3. current history is empty. 4. go to http://main.g2.bx.psu.edu/history/list_published 5. select any one of the histories (e.g. "SM_1186088") 6. Click on the green plus button to import the history. 7. The following message is displayed: History "imported: SM_1186088" has been imported. You can start using this history or return to the previous page. 8. click on "start using this history", you'll be back on the main galaxy web site, with this new history selected. So far, so good. Now, repeat steps 1-8, but instead of creating a new, empty history (step 2,3), select a non-empty history or upload some dummy data set. After clicking "start using this history" (step 8) you'll be at the *old* history, not the newly imported one. -gordon

2 2

grouping and sorting error at a bare local installation
by Jordi Rambla 02 May '10

02 May '10

Hi, I'm quite new to Galaxy and Python, but I have near 25 years of experience at computing (development, databases, web development, systems.) I've performed a local installation in order to learn Galaxy and it's "internals". In order to test the installation, I'm reproducing the Galaxy screencast tutorials locally. When reproducing the screencast named : "Finding Exons with the highest number of nucleotide polymorphisms", at step 4 (grouping and counting). I'm receiving the following error: "An error occurred running this job: Sorting input dataset resulted in error: 512: sort: multi-character tab `$\t'". Just to doublecheck, I've tried to just sort the dataset on column 4 (Name), and I'm receiving a quite similar error ("An error occurred running this job: sort: multi-character tab `$\t'"). I've tried to locate the error source at the configuration/ source code files. but found nothing promising on filters/sorter.py nor stats/grouping.py. Thus, I'm asking for a tip on where to look at. Thanks in advance. Jordi Rambla Universitat de Barcelona

3 2