Usage from Comet cluster at SDSC

Terminal

Use the foldershare command line client to upload/download data from Comet's file system.

  1. Set up your terminal environment as follows
    # You may may add the following to your ~/.bashrc file 
    export MODULEPATH=$MODULEPATH:/software/community/seedmelab/modules 
    
    # Now load the foldershare module with version noted below as follows 
    module load foldershare/1.1.0 
    
  2. Type "foldershare --help" in a terminal window to get help. 
     foldershare --help
  3. Sample data available at
    cd /software/community/seedmelab/data

Command set

You will need site's host name and your user name and password in order to connect to this site. For instance, you can list top level folders like this:

foldershare --host hostname --username myusername --password mypassword  ls /

Many Linux-style file and folder commands are supported, such as:

  • "ls" to list a folder's contents.
  • "stat" to get the status of a file or folder.
  • "mkdir" to make a new folder.
  • "rm" to remove a file or folder tree.
  • "rmdir" to remove an empty folder.
  • "cp" to copy a file or folder.
  • "mv" to move a file or folder.

There are a few more commands particular to web services, such as:

  • "download" to download a file or folder.
  • "upload" to upload a file or folder.
  • "update" to change a file or folder's description.

FolderShare shell for interactive usage

Login to the site, this will drop the user in a foldershare shell, which enables easier manual interaction

foldershare --host hostname --username myusername --password mypassword 

Create a new folder say "myfolder" at the top level (Will be available under Personal files when visited via the web browser).

foldershare> mkdir --help
foldershare> mkdir /myfolder

Upload a local file to the site using the upload command 

foldershare> upload --help
foldershare> upload /full/localpath/omega.zip /myfolder

Add a description to a file using the update command

foldershare> update --help
foldershare> update description 'this is my annotation' /myfolder/omega.zip

Download a folder using the download command, this will zip and download the contents if appropriate

foldershare> download --help
foldershare> download /myfolder /full/localpath

Scripted usage example

Create environment variables to hold the following information

# You may add the following to your ~/.bashrc file
myusername='abc' # Your username for hpcshare.sdsc.edu
mypassword='xyz' # Your password for hpcshare.sdsc.edu

# Shorthand command 
fs="foldershare --host hostname --username $myusername --password $mypassword"

Create a new folder say "myfolder" at the top level (Will be available under Personal files when visited via the web browser).

$fs mkdir /myfolder

Upload a local file to the site using the upload command 

$fs upload /full/localpath/omega.zip /myfolder

Add a description to a file using the update command

$fs update description 'this is metadata' /myfolder/omega.zip

Download a folder using the download command, this will zip and download the contents if appropriate

$fs download /myfolder /full/localpath