We are working on using [https://www.irods.org iRODS], as a way to increase storage for users of the HUB. Possible uses include simple staging of data for simulation, and housing searchable collections of documents. Some theory about our approach can be found in the paper [http://hpc2.org/resources/39 here].

We were using two servers to test iRods at CCR. The first was both a data and metadata server. The second was just a data server. Now, we upgraded to iRODS 3.0 and are only using one server.


== Setup for New User ==

Ask the administrator to create an irods account for you with a password you'd like.

File: [[File(irodsEnv)]]

Modify the user name inside the file linked above, and change the file name to ''.irodsEnv''

Create a folder name ''.irods'' in the user's home directory and place ''.irodsEnv'' in the folder. 

Create a folder named ''irods'' in your home directory. This folder will be used as the mount location for the the iRODS filesystem.

Type the command '''use irods''' in your workspace command line to be able to run icommands.

[[Image(iRods.png)]] 

To make your password persist, type '''iinit''' and enter your irods password.

Create a bash profile file name ''.bash_profile'' with the following content:

{{{
if [ -n $IRODS_MOUNT -a -e $IRODS_MOUNT ]; then
$IRODS_MOUNT
fi
}}}

Then, run the command '''source .bash_profile'''

Finally, if you type '''ls irods''' from your home directory, you should see all the users'
home folders in the irods directory, including your own.

== .bashrc files for users ==

Once icommands and irodsFs are installed into ''/apps'' the script will read similar to this:

{{{
PATH=/apps/iRODS/clients/icommands/bin:/apps/iRODS/clients/fuse/bin:$PATH
irodsFs $HOME/irods 1>/dev/null 2>&1
}}}

== submit and irods == 

Make sure ''preprocess_irods.sh'' and ''postprocess_irods.sh'' are set to '''executable'''

{{{
chmod +x preprocess_irods.sh
chmod +x postprocess_irods.sh
}}}

Also when using the ''submit'' command, be sure to include the ''preprocess_irods.sh'' and ''postprocess_irods.sh'' files

{{{
submit ... -i preprocess_irods.sh -i postprocess_irods.sh ...
}}}

If you are placing the output of a job into iRods, make sure to use the '''-q''' option with submit so that there will be no limit on the file size that you can create.


=== modification to /san/user/vhub/u2/bin/receiveinput.sh ===

{{{
IRODS_INPUT_NAME="preprocess_irods.sh"
if  [ -x ./$IRODS_INPUT_NAME ]; then
./$IRODS_INPUT_NAME
fi
}}}

=== modification to /san/user/vhub/u2/bin/transmitresults.sh ===

{{{
IRODS_OUTPUT_NAME="postprocess_irods.sh"
if [ -x ./$IRODS_OUTPUT_NAME ]; then
./$IRODS_OUTPUT_NAME
fi
}}}

== Common Problems ==

=== Client cannot connect to server ===

Server may go down unexpectedly. Admin should log onto ''irods1.ccr.buffalo.edu'', change to the iRods directory and type
 
''irodsctl'' restart

It's better to use restart than istart because sometimes the SQL server has gone down but the iRODS daemon is still running. istart will not resart the database.

Check the server log to see whether the restart succeded. The log is in iRODS/server/log
and it is named by the date, e.g. rodsLog.2010.7.11

=== vhub cannot access the users file ===

The user must do the command 

''ichmod'' inherit username

''ichmod'' -r own vhub /vhub/home/username

== Idea for submitted apps to get irods files ==

Use file browser to get path to file.

 * Parse path to get file name.

 * when submit arrives, it makes a symbolic link to the file name and name the link with the file name.

 * remove the symbolic link  after the app runs.

{{{
ln -s irods/username/pathToFile/filename filename
}}}

 * submit execs app, file "resides" in cwd.

{{{
rm filename
}}}

== To use a remote command-line client: ==


Get a copy of the file .irodsEnv from us ('''Note''': the irodsHost needs to be an '''''IP address''''' 
rather than a domain name) and
have us set up your '''login name''' and '''password''' on the
iRods server.

From your home directory on the hub:

{{{
mkdir ~/.irods
mv .irodsEnv ~/.irods
}}}

[https://www.irods.org/index.php/Downloads Download iRods]

Follow these [https://www.irods.org/index.php/Installation instructions] for '''installing additional clients'''.

At this point you will need to set your path:

{{{
PATH=home/vhub/username/iRODS/clients/icommands/bin:$PATH
}}}

It's easiest if you put this command in your ''.bashrc'' or ''.profile''
file so the commands are available every time you open a terminal. 

== Some basic concepts ==

 * '''collection''': a directory

 * '''resource''': represents a physical data server location that can hold many collections

 * '''zone''': multiple resources can be members of the same zone

== Some useful commands ==

 * '''iinit''': enter your password and it will get cahed so you don't have to give your password for every command.

 * '''ils''': list files

 * '''imkdir''': create a directory (also called a collection)

''imkdir'' username/folder

 * '''iput''': put a file into iRods

''iput'' foo.tar username/folder

 * '''ireg''': copy a directory revursively into iRods

'''NOTE''': If you add files to the actual physical directory, this is not reflected in iRODS.

{{{
ireg -C /data/iRODS/testIreg/ /tempZone/home/rods/newCol
ils newCol
/tempZone/home/rods/newCol:
   foo1
   foo2
cd testIreg/
touch bar
ils newCol
/tempZone/home/rods/newCol:
   foo1
   foo2
}}}

== iRods FUSE notes ==

Install: Need to install ''libfuse-dev'' to get the header files in the same path as
the library

edit `iRODS/config/config.mk`

{{{
uncomment IRODS_FS = 1
}}}

set ''fuseHomeDir=path/to/fuse'' where the path is just above ''include/fuse.h'' and ''lib/libfuse.a''

on your vhub account this is simply  ''fuseHomeDir=/usr''


in iRODS/client/fuse do '''make'''

Add another path so that you can mount with the command ''irodsFs''

'''NOTE: You cannot unmount on  your vhub account, once you've mounted irods.
Only a vhub dministrator can run fusermount.'''

{{{
PATH=/home/alisanee/iRods/iRODS/clients/icommands/bin:/home/alisanee/iRods/iRODS/clients/fuse/bin:$PATH
}}}

== Public share directory ==

While using iRODS you may want to share files with different users. One way to do this is to put the files you want to share in ''/vhub/home/public''. Any user that is on the iRODS system can read and write to this public directory.

''icp'' foo /vhub/home/public

'''ADMIN NOTE:''' When users are created, they are automatically added to the ''share'' group. The permissions on the ''/vhub/home/public'' directory is set up so anyone in the ''share'' group is allowed r/w access.

If you want to allow users that do not have an account on the iRODS system to access the public share directory, you need to create the ''anonymous'' user. Create the user just like all the other users, but you do not have to set the password.

''iadmin'' mkuser anonymous rodsuser

After creating the user, allow that user to ''read'' the ''/vhub/home/public'' directory.

''ichmod'' read anonymous /vhub/home/public

''ichmod'' inherit anonymous

After these steps, anyone that does not have an account on the irods system is able to read the files located in the ''/vhub/home/public'' directory. They are not able to write or go up the directory structure.

In order for users to login to iRODS with this ''anonymous'' user, all you have to do is use the username ''anonymous'' with any password.

The username ''anonymous'' is a special account name that is recognised in the iRODS system.

Admin's can use the following script to automate the process of creating the necessary files needed to use the ''anonymous'' user in vhub and mount the ''/vhub/home/public'' directory on iRODS to ''~/irods'' directory on the users filesystem on vhub.

File : [[File(irods_anonymous.sh)]]

== Administrators' To Do List For Installing on HUB ==

put the icommands in the path for every user

install .irods directory for every user and provide an irodsEnv file with their Hub ID as the user name

do iinit once with their user name and password. This will create a file, .irods/.irodsA which holds the user's encrypted password, so that they won't have to enter their password to do an icoomand.

in irods, create a home directory for the user and set the group to the user name.

irodsFs mount in an empty folder named irods

'''irodsFs irods'''

===For New User===
As the 'rods' administrator, for a new hub user: 

'''iadmin mkuser username#vhub rodsuser''' ''create new user (use rodsadmin if they are an admin)''

'''iadmin moduser username password somepassword ''' ''set user's password''

~~'''imkdir username''' ''make home directory for new user''~~

~~'''ichmod -V own username username''' ''let user rwx her home directory''~~

'''ichmod read vhub /home/username'''       ''let vhub rwx user's home directory''

'''ichmod write vhub /home/username'''

'''ichmod inherit username'''       ''let vhub rwx the subdirectories and files the user creates''

'''NOTE:''' if you are creating a ''rodsadmin'' account, then you must use the '''-M''' option with ichmod

users can then do 

''iput'' <file> <directory>

''iput'' foo username

Note that the ''irods1.ccr.buffalo.edu'' name is not available to a DNS server, so until that
changes, the irodsEnv has to be edited to have the IP address instead.

Edit `.bashrc` in iRods server(done):

{{{
PATH=/data/iRODS/iRODS/clients/icommands/bin:$PATH
}}}

Install iRods on vhub@u2-grid (done)

edit `.bashrc`

{{{
PATH=/san/user/vhub/u2/iRODS/clients/icommands/bin:$PATH
}}}

''iadmin'' mkuser username rodsuser 
 
''iadmin'' mkuser vhub rodsadmin

===Transferring large files===

When you send or receive a file to/from iRODS that's over 32 MB, it starts opening multiple ports so it can multithread the data transfers. You need to make sure the correct ports are open or the data transfer will not work.

Open ports ''20,000'' to ''20,199'' in the firewall

For irods@irods1.ccr.buffalo.edu edit `~/iRODS/config/irods.config` to say

{{{
$SVR_PORT_RANGE_START = '20000';
$SVR_PORT_RANGE_END = '20199';
}}}

and also

`~/iRODS/scripts/perl/irodsctl.pl`

to say

{{{
$svrPortRangeStart=20000;
$svrPortRangeEnd=20199;
}}}

Then restart the server

''irodsctl'' restart

'''Note:''' when doing the multithreaded data transfers we had a problem with the server only listening on the internal IP address.

{{{
> iput -V -f test_dir_timing_40mb_80files.tar kmarcus/test_time
NOTICE: irodsHost=irods1.ccr.buffalo.edu
NOTICE: irodsPort=1247
NOTICE: irodsDefResource=demoResc
NOTICE: irodsHome=/vhub/home
NOTICE: irodsCwd=/vhub/home
NOTICE: irodsUserName=kmarcus
NOTICE: irodsZone=vhub
>From server: NumThreads=2, addr:10.6.1.20, port:20169,
cookie=810769104
ERROR: connectToRhostPortal: connectTo Rhost 10.6.1.20 port 20169
error, status = -347000
}}}

'''Fix:''' edit `~/iRODS/server/config/irodsHost` to say:

{{{
localhost irods1.ccr.buffalo.edu 10.6.1.20
}}}

This tells iRODS to prefer the interface ''irods1.ccr.buffalo.edu'' when binding.

Information for this fix was provided by Chris Smith on [http://groups.google.com/group/irod-chat/browse_thread/thread/24ade1a0b1954d42/a39ad7965695e5d1?lnk=gst&q=kmarcus30#a39ad7965695e5d1 iRODS Chat]

==Uninstalling iRODS server==

remove the collections (i.e. directories and files), e.g.

'''irm  -r aneeman'''

turn irods off 

'''irodsctl stop'''

remove directories for irods and its postgres installation.

==Changing the server==

===Rename resource===

{{{
iadmin modresc oldName name newName
}}}

edit config/irods.config

{{{
$RESOURCE_NAME = 'newName';
}}}

restart the server

{{{
./irodsctl restart
}}}

===Change hostname===

edit config/irods.config

{{{
$DATABASE_HOST = 'new.hostname';
}}}

edit server/config/irodsHost

{{{
localhost irods.ccr.buffalo.edu 10.122.1.8
}}}

== Move a collection or file to a specific server ==

'''iphymv -r resource2Collection -R demoResc2'''

''-r'' recursive
''-R'' resource name

The "mount" always looks the same, regardless of the physical location.
You can move a file to any data server.

==iRods Timing==

[[Image(irods_timing.png)]]