Usage from Comet cluster at SDSC

Terminal

Use the foldershare command line client to upload/download data from Comet's file system.

  1. Set up your terminal environment as follows
    # You may may add the following to your ~/.bashrc file 
    export MODULEPATH=$MODULEPATH:/software/community/seedmelab/modules 
    
    # Now load the foldershare module with version noted below as follows 
    module load foldershare/1.2.0 
    
  2. Type "foldershare --help" in a terminal window to get help. 
     foldershare --help
  3. Sample data available at
    cd /software/community/seedmelab/data

FolderShare client command set

You will need this temporary site's host name and your user name and password in order to connect to this site. For instance, you can list top level folders like this:

./foldershare --host hostname --username login --password password  ls /

Many Linux-style file and folder commands are supported, such as:

  • "ls" to list a folder's contents.
  • "stat" to get the status of a file or folder.
  • "mkdir" to make a new folder.
  • "rm" to remove a file or folder tree.
  • "rmdir" to remove an empty folder.
  • "cp" to copy a file or folder.
  • "mv" to move a file or folder.

There are a few more commands particular to web services, such as:

  • "download" to download a file or folder.
  • "upload" to upload a file or folder.
  • "update" to add or change a file or folder's description.

FolderShare shell for interactive usage

Login to the site, this will drop the user in a foldershare shell, which enables easier manual interaction.

./foldershare --host hostname \
              --username login \
              --password password 

Create a new folder say "myfolder" at the top level (Will be available under Personal files when visited via the web browser).

foldershare> mkdir --help
foldershare> mkdir /myfolder

Upload a local file to the site using the "upload" command .

foldershare> upload --help

# Upload a file showing progress and preserving local timestamps
foldershare> upload -v -p /full/localpath/omega.zip /myfolder

# Upload missing files from local folder to remote destination 
foldershare> upload -v --sync /full/localpath/omega.zip /myfolder

Add a description to a file using the "update" command.

foldershare> update --help

# Add a description to a file
foldershare> update description 'this is metadata' /myfolder/omega.zip

Download a folder using the "download" command, this download the contents.

foldershare> download --help

# Download remote folder with progress display and preserve timestamps
foldershare> download -v -p /myfolder /full/localpath

# Download missing local files/folders from remote destination
foldershare> download -v -p /myfolder /full/localpath

 

Scripted usage example

Create a variable to hold the following information.

prefix="./foldershare --host hostname --username login --password password"

Create a new folder say "myfolder" at the top level (Will be available under Personal files when visited via the web browser).

$prefix mkdir /myfolder

Upload a local file to the site using the upload command.

$prefix upload -v -p /full/localpath/omega.zip /myfolder

Add a description to a file using the update command.

$prefix update description 'this is metadata' /myfolder/omega.zip

Download a folder using the "download" command, this download the contents.

$prefix download -v -p /myfolder /full/localpath