Difference: MassSe (11 vs. 12)

Revision 122010-05-07 - AndrasLaszlo

Line: 1 to 1
 
META TOPICPARENT name="WebHome"

Mass file handling on the Grid (the GSTREAM library)

Added:
>
>

Installation

 Note: The GSTREAM library is installed on the KFKI AFS. Look at that page for instructions on activation.
Added:
>
>
To install it to your computer, download GSTREAM, then say:
tar  -xzf  gstream.tar.gz
cd  gstream
./configure  [--prefix=some_installation_path]
make
make  install
 

The GSTREAM library for read/write C++ streams to Storage Elements

As one does generally not want to always stage out the data files from SE-s onto a local disk by hand, and then process it, it is recommended to have read/write streams. The C++ library GSTREAM implements such a library. ( gstream, igstream, ogstream stream classes, like the usual C++ STL fstream, ifstream, ofstream file input/output stream classes; the letter 'g' standing for 'grid'.) It does nothing else, but treats the file as a normal file, unless its name begins with the string /grid/. In this case, it stages out the datafile in question onto a local (or AFS) area, and then treats the local file as a normal file. Sooner or later this solution has to be replaced with a GFAL based C++ library.

Line: 17 to 28
  int main(int argc, char *argv[]) {
Added:
>
>
// Set debug and checksum levels for verifying what is happening (default: 0, i.e. don't check). gstream_debug(1); gstream_checksum(1);
  // Open the datafile for reading. igstream igfile("/grid/cms/alaszlo/some_datafile.dat"); // Extract data from your datafile with 'igstream::operator>>' or with 'igstream::read(char*, int)'.
Line: 93 to 108
  The wrapper scripts may be improved in the future. We thank to Kálmán Kővári for this toolkit. There are also other useful scripts in the package. Please have a look a them.
Changed:
<
<
Also the scripts gexists.sh, gput.sh, grep.sh, gget.sh, gdel.sh may be helpful to manipulate single files.
>
>
Also the scripts gexists.sh, gput.sh, grep.sh, gget.sh, gdel.sh, gcheck.sh may be helpful to manipulate single files.
 

-- AndrasLaszlo - 10 Jul 2009

Changed:
<
<
META FILEATTACHMENT attachment="gstream.tar.gz" attr="" comment="The GSTREAM library source." date="1273160081" name="gstream.tar.gz" path="gstream.tar.gz" size="20887" stream="gstream.tar.gz" tmpFilename="/usr/tmp/CGItemp5148" user="AndrasLaszlo" version="6"
>
>
META FILEATTACHMENT attachment="gstream.tar.gz" attr="" comment="The GSTREAM library source." date="1273224241" name="gstream.tar.gz" path="gstream.tar.gz" size="20879" stream="gstream.tar.gz" tmpFilename="/usr/tmp/CGItemp6887" user="AndrasLaszlo" version="7"
 
This site is powered by the TWiki collaboration platform Powered by Perl This site is powered by the TWiki collaboration platformCopyright &© by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback