- galaxy-dev - lists.galaxyproject.org

Problem uploading multigigabyte files via API
by Sytchev, Ilya 27 Apr '16

27 Apr '16

Hi, I'm running Galaxy 16.01 behind Nginx 1.4.6 on Ubuntu 14.04 and I came across an issue where uploading files larger than ~3GB via the API into histories and libraries fails. Steps to reproduce (using BioBlend 0.7.0): from bioblend.galaxy.objects import GalaxyInstance gi = GalaxyInstance("galaxy-dev", "api_key") h = gi.histories.create() h.upload_dataset('/path/to/multigigabytefile') Observed results: ConnectionError: Unexpected response from galaxy: 504: <html> <head><title>504 Gateway Time-out</title></head> <body bgcolor="white"> <center><h1>504 Gateway Time-out</h1></center> <hr><center>nginx/1.4.6 (Ubuntu)</center> </body> </html> It looks like Nginx is accepting the full incoming data transfer over the network and saving it to disk successfully, and the error occurs during the subsequent forwarding of the request to Galaxy. There doesn't seem to be a particular threshold for file size (for example, 2GB and 2.7GB file can be uploaded successfully). Also, large file uploads do work when Galaxy is not running behind Nginx. Relevant log entries and config details are attached. I'd appreciate any help. Thanks, Ilya

1 0

output datasets
by Ryan G 27 Apr '16

27 Apr '16

I'm playing around with creating a Galaxy wrapper for custom tool I've created. My tool takes a single parameter, and SRA ID. In command block, I have something like this: <command> ${GALAXY_ROOT_DIR}/tools/mycustomtools/doSomething.pl --sra $sraid </command> <inputs> <param name="sraid" ... /> </inputs> <outputs> <data format="txt", name="out1", from_work_dir="${sraid}.data" /> <data format="txt", name="out2", from_work_dir="${sraid}.txt" /> </outputs> Galaxy doesn't like thr $sraid in the output. I see from the docs I need to use discover_datasets. It would be nice to use a parameter such as what I have to name output file as my script names output files use an input parameter as the base.

2 2

PSU Meet and Greet this Saturday
by Suderman Keith 23 Apr '16

23 Apr '16

Sorry for spamming the list, but I thought this would be the easiest/quickest way to reach out to any interested people in State College. I had to make a short-notice trip to meet with colleagues in Pittsburgh and I will be driving home to NY Saturday. Since I am driving right by State College I thought it would be silly, if not downright rude, if I didn’t at least ask if anyone is up for a quick meet and greet somewhere? I should be passing by around 1-2PM. I would like to put some faces to the names I see on the list and you guys might be interested in learning more about what the NLP community is doing with Galaxy. Cheers, Keith ------------------------------ Research Associate Department of Computer Science Vassar College Poughkeepsie, NY

2 2

ERROR: PIP is not a legl parament of an Ansible Play
by William Ge 21 Apr '16

21 Apr '16

Hi, all, Sorry to bother you again. I am trying to install tools using the script. I have downloaded the playbook, but stuck here: [galaxy@Pegasus ansible-galaxy-tools]$ ansible-playbook tasks/tools.yml -i "localhost," --extra-vars galaxy_tools_api_key=xcd2Glx ERROR: pip is not a legal parameter of an Ansible Play I did not find anything related on the Galaxy search either. Any help will be appreciated. The web Admin interface only allows tool installation one by one. I wish it allows batch installation there. Thanks, Bill

4 20

Main Toolshed Account password reset
by Oksana Korol 21 Apr '16

21 Apr '16

Sorry for the previous email's subject line. Re-sending the questions with the proper subject. Hi, Who is the administrator on Galaxy Main Toolshed ( https://toolshed.g2.bx.psu.edu)? I'm trying to re-set a password for the work Galaxy toolshed account, but I think that the email is blocked by our firewall. Who can I contact to resolve this issue? Thanks! -- :: Oksana Korol

1 0

Re: [galaxy-dev] galaxy-dev Digest, Vol 118, Issue 18
by Oksana Korol 21 Apr '16

21 Apr '16

Hi, Who is the administrator on Galaxy Main Toolshed ( https://toolshed.g2.bx.psu.edu)? I'm trying to re-set a password for the work Galaxy toolshed account, but I think that the email is blocked by our firewall. Who can I contact to resolve this issue? Thanks! Oksana Korol

1 0

GalaxyAdmins Meetup THIS THURSDAY: Galaxy on Mobile Devices; and Tools, Tools, Tools!
by Hotz, Hans-Rudolf 21 Apr '16

21 Apr '16

Hello all, The next <https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2016_04_21> GalaxyAdmins<https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2016_04_21> online meetup<https://wiki.galaxyproject.org/Community/GalaxyAdmins/Meetups/2016_04_21> is this Thursday, April 21 at 5PM CET (see your local time<http://bit.ly/1RpBa8e> to know when to connect): The featured topics are: The Galaxy Portal: Accessing Galaxy from Mobile Devices<http://bioinformatics.oxfordjournals.org/content/early/2016/02/17/bioinform…> Ivar Grytten<http://www.mn.uio.no/ifi/english/people/aca/ivargry/> and Geir Kjetil Sandve<http://www.mn.uio.no/ifi/english/people/aca/geirksa/>, University of Oslo<http://www.uio.no/> Tool Development Developments John Chilton<https://wiki.galaxyproject.org/JohnChilton>, Penn State University<http://psu.edu/> The meetup will use the same Adobe Connect technology used on previous calls. The 'Meeting Room Link' will be provided on the wiki page in time. It takes a couple of minutes to connect, so you are encouraged to start connecting a few minutes early. See http://bit.ly/gxyadm201604 for full details. Thanks, Dave C and Hans-Rudolf ---- GalaxyAdmins<https://wiki.galaxyproject.org/Community/GalaxyAdmins> is a discussion group for Galaxy community members who are responsible for Galaxy installations.

3 2

FTP Authentication Against Postgres Database with PBKDF2
by Joseph Brent Greer 20 Apr '16

20 Apr '16

Hi All, We're having problems setting up ftp and authenticating against the postgres database. We've reviewed these pages and a few others: Galaxy Page: https://wiki.galaxyproject.org/Admin/Config/UploadviaFTP Peter Briggs Blog Post: http://galacticengineer.blogspot.co.uk/2015/02/ftp-upload-to-galaxy-using-p… Ricardo Perez's Galaxy Developers thread that Peter referenced: http://dev.list.galaxyproject.org/ProFTPD-integration-with-Galaxy-td4660295… When we run proftpd in debug mode we get similar output to the one in Ricardo's thread: Ricardo's command line output: login002 proftpd[987] login002 (10.28.56.101[10.28.56.101]): mod_sql_passwd/0.4: expected 'PBKDF2$sha256$10000$8h/4HmD1Eu6NTc7F$Slb1H5a9YJvR6A3cUnZCUfh7tOWKfRuh', got 'cc0ef515d684386aaa500bf6499d9f512dfb332c' Our output: 2016-04-20 10:53:42,597 pcegalaxy proftpd[18649] pcegalaxy.ci.northwestern.edu (lagunitas.kelleher.northwestern.edu[129.105.112.217]): mod_sql_passwd/0.7: expected 'NMmACzgvs8Oh+BCnSw+2GFxpYl7cLiQu', got 'g3L81voHwAEh8y0B9qBImZycWgS0w4Z4' >From looking at the galaxy_user table, it seems like proftpd is finding the correct password information (password in table: PBKDF2$sha256$10000$kgyVpO6N2rWAbeX6$NMmACzgvs8Oh+BCnSw+2GFxpYl7cLiQu). The last post from that thread mentions back-channel advice that was helpful. Maybe we need the same advice ? In case it's important we're using LDAP for our general user authentication. Here is our proftpd.conf: ServerName "Galaxy FTP File Upload" ServerType standalone DefaultServer on PidFile /opt/apps/proftpd/1.3.5/var/proftpd.pid # Port 21 is the standard FTP port. Port 21 # Don't use IPv6 support by default. UseIPv6 off # Umask 022 is a good standard umask to prevent new dirs and files # from being group and world writable. Umask 077 # To prevent DoS attacks, set the maximum number of child processes # to 30. If you need to allow more than 30 concurrent connections # at once, simply increase this value. Note that this ONLY works # in standalone mode, in inetd mode you should use an inetd server # that allows you to limit maximum number of processes per service # (such as xinetd). MaxInstances 30 # Set the user and group under which the server will run. User nobody Group nobody # To cause every FTP user to be "jailed" (chrooted) into their home # directory, uncomment this line. DefaultRoot ~ # Automatically create home directory if it doesn't exist CreateHome on dirmode 700 # Allow users to overwrite their files AllowOverwrite on # Allow users to resume interrupted uploads AllowStoreRestart on # Bar use of SITE CHMOD by default <Limit SITE_CHMOD> DenyAll </Limit> # Bar use of RETR (download) since this is not a public file drop <Limit RETR> DenyAll </Limit> # Do not authenticate against real (system) users AuthPAM off ## Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1 SQLPasswordEngine on ## Set this if Galaxy user UID and/or GID are less than 999 SQLMinID 400 ## Configuration that handles PBKDF2 encryption ## Set up mod_sql to authenticate against the Galaxy database SQLEngine on SQLBackend postgres SQLConnectInfo galaxy_db@localhost:5432 <USERNAME> <PASSWORD> SQLAuthTypes PBKDF2 SQLPasswordPBKDF2 SHA256 10000 24 SQLPasswordEncoding base64 SQLAuthenticate users ## For PBKDF2 authentication SQLPasswordUserSalt sql:/GetUserSalt ## Define a custom query for lookup that returns a passwd-like entry for PBKFD2 ## UID and GID should match your Galaxy user. SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT "email, (CASE WHEN substring(password from 1 for 6) = 'PBKDF2' THEN substring(password from 38 for 69) ELSE password END) AS password2,400,400,'/share/PCEitAdmin/Galaxy/external_users/%U','/bin/bash' FROM galaxy_user WHERE email='%U'" ## Define custom query to fetch the password salt SQLNamedQuery GetUserSalt SELECT "(CASE WHEN SUBSTRING (password from 1 for 6) = 'PBKDF2' THEN SUBSTRING (password from 21 for 16) END) AS salt FROM galaxy_user WHERE email='%U'" Any advice would be great. Thanks, Joe Greer Northwestern University

1 0

galaxy running problem
by V.Freire-Beneitez 20 Apr '16

20 Apr '16

Dear Sir or Madam, Since yesterday I have been trying to run some RNA-sequencing analysis using the tools from galaxy, but it seems that it is frozen not even sending to queue the runnings and the tophats are not working. The webpage says that it is being updated and there could be some problems. I would like to ask you if more people is encountering this issue and if there is any way I could solve it. Thanks a lot for your time and help Regards, Veronica

2 1

set_metadata in galaxy_XXX.sh script
by Laure QUINTRIC 20 Apr '16

20 Apr '16

Hi there (sorry for all theses questions), when I execute a job with remote user drmaa configuration on our pbs cluster, I see in the job working directory the script the galaxy_XXX.sh which set the galaxy virtual env and execute set_metadata.py after tool_script.sh completion but I get this in cluster error output : python: /lib64/libz.so.1: no version information available (required by python) Could not find platform dependent libraries <exec_prefix> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>] Traceback (most recent call last): File "galaxy/database/job_working_directory/002/2824/set_metadata_tkCRRk.py", line 1, in <module> from galaxy_ext.metadata.set_metadata import set_metadata; set_metadata() File "galaxy/galaxy/lib/galaxy_ext/metadata/set_metadata.py", line 15, in <module> import json ImportError: No module named json The tool is execution is ok but the metadata script failed, but as that script is executed on the cluster and not on galaxy server, (I think) the virtual env is not properly set. Is there a way to remove set_metadata from galaxy_XXX.sh script as It was in v16.01 ? I don't really understand what to set in galaxy.ini (I tried to set retry_metadata_internally to True and then False but I get the same result). Regards, Laure

1 0